2024-05-08T05:30:07.095 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T05:30:07.096 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --name vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} --verbose --archive /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246 -- /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/orig.config.yaml 2024-05-08T05:30:07.151 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:30:07.249 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246 branch: wip-57676-3 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} email: vshankar@redhat.com first_in_suite: false job_id: '7697246' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-57676-3 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-57676-3 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5998 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a sleep_before_teardown: 0 subset: 915/1024 suite: fs:workload suite_branch: wip-57676-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi003.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/DbRa776XN7dGQLDnT/RJV2PFz++YF2U5KSitaeSOrSLTjgEhjlZHk02QWUZX72OgCn5DAi6qwVwIg8KVvVOg= smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4SzUhmcW/0uuurypAK7oId25cwOoiSr8oYwF88N9Wms1/1jxSiM1ZWwr3FOPlhwKHf41ZMb7d7t/gc5WDd2js= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGetSfF2dFYoqkCPUV4FX5KrXk03nRnLMaASmDYiaDMMkoZayO+GTCpQQjaS6j+H0y5wYltMmHhBpbip4BG5ec= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsync-tester.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_04:00:39 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T05:30:07.249 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa; will attempt to use it 2024-05-08T05:30:07.249 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks 2024-05-08T05:30:07.250 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T05:30:07.251 INFO:teuthology.task.internal:Checking packages... 2024-05-08T05:30:07.274 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a' 2024-05-08T05:30:07.274 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T05:30:07.274 INFO:teuthology.packaging:ref: None 2024-05-08T05:30:07.274 INFO:teuthology.packaging:tag: None 2024-05-08T05:30:07.274 INFO:teuthology.packaging:branch: wip-57676-3 2024-05-08T05:30:07.274 INFO:teuthology.packaging:sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:30:07.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-57676-3 2024-05-08T05:30:07.491 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3538.gbee299f4 2024-05-08T05:30:07.492 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T05:30:07.499 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T05:30:07.499 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T05:30:07.543 INFO:teuthology.task.internal:Saving configuration 2024-05-08T05:30:07.564 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T05:30:07.570 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:30:07.595 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:04.761758', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/DbRa776XN7dGQLDnT/RJV2PFz++YF2U5KSitaeSOrSLTjgEhjlZHk02QWUZX72OgCn5DAi6qwVwIg8KVvVOg='} 2024-05-08T05:30:07.612 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:04.763745', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4SzUhmcW/0uuurypAK7oId25cwOoiSr8oYwF88N9Wms1/1jxSiM1ZWwr3FOPlhwKHf41ZMb7d7t/gc5WDd2js='} 2024-05-08T05:30:07.632 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:04.765566', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGetSfF2dFYoqkCPUV4FX5KrXk03nRnLMaASmDYiaDMMkoZayO+GTCpQQjaS6j+H0y5wYltMmHhBpbip4BG5ec='} 2024-05-08T05:30:07.632 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T05:30:07.639 INFO:teuthology.task.internal:roles: ubuntu@smithi003.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T05:30:07.639 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T05:30:07.639 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T05:30:07.639 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T05:30:07.746 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0c268fc0d0>, signals=[15]) 2024-05-08T05:30:07.747 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T05:30:07.753 INFO:teuthology.task.internal:Opening connections... 2024-05-08T05:30:07.753 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:30:07.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:30:07.837 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-05-08T05:30:07.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:30:07.914 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-05-08T05:30:07.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:30:07.989 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T05:30:07.996 DEBUG:teuthology.orchestra.run.smithi003:> uname -m 2024-05-08T05:30:08.015 INFO:teuthology.orchestra.run.smithi003.stdout:x86_64 2024-05-08T05:30:08.015 DEBUG:teuthology.orchestra.run.smithi003:> cat /etc/os-release 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:NAME="CentOS Stream" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION="9" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:ID="centos" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:VERSION_ID="9" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:ANSI_COLOR="0;31" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:30:08.073 INFO:teuthology.orchestra.run.smithi003.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:30:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:30:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:30:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:30:08.074 INFO:teuthology.orchestra.run.smithi003.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:30:08.074 INFO:teuthology.lock.ops:Updating smithi003.front.sepia.ceph.com on lock server 2024-05-08T05:30:08.096 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-05-08T05:30:08.114 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-05-08T05:30:08.114 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:30:08.171 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-05-08T05:30:08.172 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:30:08.172 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:30:08.172 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:30:08.172 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:30:08.172 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:30:08.172 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:30:08.172 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-05-08T05:30:08.196 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-05-08T05:30:08.214 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-05-08T05:30:08.214 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-05-08T05:30:08.271 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-05-08T05:30:08.271 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-05-08T05:30:08.271 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-05-08T05:30:08.271 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:30:08.271 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:30:08.272 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:30:08.272 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-05-08T05:30:08.296 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T05:30:08.367 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T05:30:08.373 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T05:30:08.373 DEBUG:teuthology.orchestra.run.smithi003:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:30:08.375 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:30:08.378 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:30:08.394 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T05:30:08.400 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T05:30:08.400 DEBUG:teuthology.orchestra.run.smithi003:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:30:08.433 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:30:08.436 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:30:08.461 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T05:30:08.536 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T05:30:08.551 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-08T05:30:08.551 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T05:30:08.551 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T05:30:08.551 DEBUG:teuthology.orchestra.run.smithi003:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:30:08.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:30:08.569 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-05-08T05:30:08.627 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-437.el9.x86_64 2024-05-08T05:30:08.627 INFO:teuthology.task.kernel:Running kernel on smithi003: 5.14.0-437.el9.x86_64 2024-05-08T05:30:08.627 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum install -y kernel 2024-05-08T05:30:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.1 MB 00:03 2024-05-08T05:30:19.404 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 19 MB 00:04 2024-05-08T05:30:23.930 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-05-08T05:30:25.789 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-05-08T05:30:26.799 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-08T05:30:32.124 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-08T05:30:34.263 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T05:30:34.264 INFO:teuthology.orchestra.run.smithi003.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T05:30:34.348 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-05-08T05:30:34.364 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout: Package Architecture Version Repository Size 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout:================================================================================ 2024-05-08T05:30:34.365 INFO:teuthology.orchestra.run.smithi003.stdout:Install 4 Packages 2024-05-08T05:30:34.366 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:30:34.366 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 78 M 2024-05-08T05:30:34.366 INFO:teuthology.orchestra.run.smithi003.stdout:Installed size: 123 M 2024-05-08T05:30:34.366 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-05-08T05:30:34.775 INFO:teuthology.orchestra.run.smithi003.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.8 MB/s | 484 kB 00:00 2024-05-08T05:30:35.558 INFO:teuthology.orchestra.run.smithi003.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 15 MB/s | 15 MB 00:01 2024-05-08T05:30:36.192 INFO:teuthology.orchestra.run.smithi003.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 20 MB/s | 34 MB 00:01 2024-05-08T05:30:36.576 INFO:teuthology.orchestra.run.smithi003.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 16 MB/s | 28 MB 00:01 2024-05-08T05:30:36.576 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:30:36.576 INFO:teuthology.orchestra.run.smithi003.stdout:Total 35 MB/s | 78 MB 00:02 2024-05-08T05:30:37.282 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-05-08T05:30:37.714 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-05-08T05:30:37.714 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-05-08T05:30:38.691 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-05-08T05:30:38.692 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-05-08T05:30:41.217 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-05-08T05:30:41.867 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:30:41.951 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:30:44.749 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:30:44.879 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:30:49.967 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:30:50.006 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:30:55.197 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:26.274 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:26.313 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:26.621 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:26.621 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:31:26.621 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:31:26.621 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:31:27.397 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:31:27.397 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:31:27.397 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-05-08T05:31:27.397 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T05:31:27.397 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T05:31:27.397 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T05:31:27.398 INFO:teuthology.orchestra.run.smithi003.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T05:31:27.398 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:31:27.398 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-05-08T05:31:27.739 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:31:27.821 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:31:27.821 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:31:27.821 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T05:31:27.821 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T05:31:27.821 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T05:31:27.821 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-08T05:31:27.821 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi003.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T05:31:27.822 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:31:27.872 INFO:teuthology.orchestra.run.smithi003.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:31:27.872 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:31:27.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo rpm -qi grub2-tools 2024-05-08T05:31:27.905 INFO:teuthology.orchestra.run.smithi003.stdout:Name : grub2-tools 2024-05-08T05:31:27.905 INFO:teuthology.orchestra.run.smithi003.stdout:Epoch : 1 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Version : 2.06 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Release : 68.el9 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Architecture: x86_64 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Group : Unspecified 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Size : 8264254 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:License : GPLv3+ 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Packager : builder@centos.org 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:Vendor : CentOS 2024-05-08T05:31:27.906 INFO:teuthology.orchestra.run.smithi003.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:Summary : Support tools for GRUB. 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:Description : 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:hardware devices. 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:31:27.907 INFO:teuthology.orchestra.run.smithi003.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T05:31:27.908 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T05:31:27.908 INFO:teuthology.task.kernel:Updating grub on smithi003 to boot 5.14.0-444.el9.x86_64 2024-05-08T05:31:27.908 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T05:31:28.507 INFO:teuthology.orchestra.run.smithi003.stderr:Generating grub configuration file ... 2024-05-08T05:31:30.226 INFO:teuthology.orchestra.run.smithi003.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T05:31:30.275 INFO:teuthology.orchestra.run.smithi003.stderr:done 2024-05-08T05:31:30.287 DEBUG:teuthology.orchestra.run.smithi003:> mktemp 2024-05-08T05:31:30.309 INFO:teuthology.orchestra.run.smithi003.stdout:/tmp/tmp.tO6H6Lrgex 2024-05-08T05:31:30.310 DEBUG:teuthology.orchestra.run.smithi003:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.tO6H6Lrgex 2024-05-08T05:31:30.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0666 /tmp/tmp.tO6H6Lrgex 2024-05-08T05:31:30.521 DEBUG:teuthology.orchestra.remote:smithi003:/tmp/tmp.tO6H6Lrgex is 6KB 2024-05-08T05:31:30.543 DEBUG:teuthology.orchestra.run.smithi003:> rm -fr /tmp/tmp.tO6H6Lrgex 2024-05-08T05:31:30.557 DEBUG:teuthology.orchestra.run.smithi003:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T05:31:30.619 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T05:31:30.620 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T05:31:30.620 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T05:31:30.620 INFO:teuthology.orchestra.run.smithi003.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T05:31:30.621 DEBUG:teuthology.orchestra.run.smithi003:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T05:31:30.901 DEBUG:teuthology.orchestra.run.smithi003:> sudo shutdown -r now 2024-05-08T05:32:00.904 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T05:32:00.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-08T05:32:00.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:32:19.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.3 2024-05-08T05:32:28.360 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-08T05:32:28.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:33:28.363 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T05:33:40.365 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-08T05:33:40.366 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:33:40.752 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-05-08T05:33:41.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-08T05:33:41.122 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-08T05:33:41.123 DEBUG:teuthology.orchestra.run.smithi003:> uname -r 2024-05-08T05:33:41.139 INFO:teuthology.orchestra.run.smithi003.stdout:5.14.0-444.el9.x86_64 2024-05-08T05:33:41.140 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T05:33:41.140 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T05:33:41.140 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T05:33:42.141 DEBUG:teuthology.parallel:result is None 2024-05-08T05:33:42.142 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T05:33:42.152 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T05:33:42.152 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:33:42.155 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:33:42.157 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:33:42.176 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T05:33:42.183 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T05:33:42.191 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T05:33:42.191 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:33:42.219 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:33:42.221 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:33:42.331 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T05:33:42.338 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T05:33:42.338 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:33:42.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:33:42.355 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:33:42.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:33:42.372 DEBUG:teuthology.orchestra.run.smithi003:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:33:42.374 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:33:42.398 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:33:42.445 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:42.452 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:42.454 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:42.471 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:42.474 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:42.495 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:33:42.497 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T05:33:42.504 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T05:33:42.505 DEBUG:teuthology.orchestra.run.smithi003:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:33:42.507 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:33:42.510 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:33:42.541 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T05:33:42.552 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T05:33:42.552 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:33:42.578 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:33:42.580 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:33:42.597 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:42.680 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:42.736 DEBUG:teuthology.orchestra.run.smithi003:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:42.819 DEBUG:teuthology.orchestra.run.smithi003:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:42.867 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:33:42.867 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:33:42.936 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:42.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:43.030 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:43.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:43.155 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:33:43.155 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:33:43.221 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:43.267 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:33:43.325 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:43.367 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:33:43.414 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:33:43.414 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:33:43.485 DEBUG:teuthology.orchestra.run.smithi003:> sudo service rsyslog restart 2024-05-08T05:33:43.488 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-05-08T05:33:43.491 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-05-08T05:33:43.526 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:33:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:33:43.559 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:33:43.948 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T05:33:43.957 INFO:teuthology.task.internal:Starting timer... 2024-05-08T05:33:43.957 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T05:33:44.043 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T05:33:44.052 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-08T05:33:44.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo service auditd rotate 2024-05-08T05:33:44.107 INFO:teuthology.orchestra.run.smithi003.stdout:Rotating logs: 2024-05-08T05:33:44.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-05-08T05:33:44.171 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-05-08T05:33:44.173 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-05-08T05:33:44.225 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-05-08T05:33:44.228 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T05:33:44.228 DEBUG:teuthology.orchestra.run.smithi003:> /usr/sbin/getenforce 2024-05-08T05:33:44.251 INFO:teuthology.orchestra.run.smithi003.stdout:Permissive 2024-05-08T05:33:44.251 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-05-08T05:33:44.274 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-05-08T05:33:44.274 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-05-08T05:33:44.299 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-05-08T05:33:44.299 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi003.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive'} 2024-05-08T05:33:44.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:33:44.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:33:44.329 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:33:44.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:33:44.356 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:33:44.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:33:44.380 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T05:33:44.381 DEBUG:teuthology.orchestra.run.smithi003:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:33:44.409 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:33:44.434 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:33:44.458 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T05:33:44.468 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T05:33:44.489 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T05:33:44.491 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi003.front.sepia.ceph.com,smithi089.front.sepia.ceph.com,smithi104.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T05:37:59.240 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi003.front.sepia.ceph.com'), Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com')] 2024-05-08T05:37:59.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-08T05:37:59.242 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi003.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:37:59.323 DEBUG:teuthology.orchestra.run.smithi003:> true 2024-05-08T05:37:59.407 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi003.front.sepia.ceph.com' 2024-05-08T05:37:59.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-05-08T05:37:59.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:37:59.483 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-05-08T05:37:59.565 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-05-08T05:37:59.565 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-08T05:37:59.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:37:59.640 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-05-08T05:37:59.726 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-05-08T05:37:59.727 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T05:37:59.738 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T05:37:59.738 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:37:59.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:37:59.741 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:37:59.741 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:37:59.743 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:37:59.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:37:59.768 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:37:59.774 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:37:59.781 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:37:59.791 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:37:59.796 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:37:59.803 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-05-08T05:37:59.810 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:37:59.814 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-05-08T05:37:59.819 INFO:teuthology.orchestra.run.smithi003.stderr:sudo: ntpd: command not found 2024-05-08T05:37:59.827 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:37:59.831 INFO:teuthology.orchestra.run.smithi003.stdout:506 Cannot talk to daemon 2024-05-08T05:37:59.833 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-05-08T05:37:59.839 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:37:59.843 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-05-08T05:37:59.847 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:37:59.856 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:37:59.862 INFO:teuthology.orchestra.run.smithi003.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:37:59.869 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:37:59.893 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:37:59.896 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:37:59.896 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-05-08T05:37:59.896 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.896 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.896 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.896 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.919 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:37:59.922 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:37:59.922 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-05-08T05:37:59.922 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.922 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.922 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.922 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.923 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:37:59.926 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:37:59.926 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-05-08T05:37:59.926 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.926 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.926 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.926 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:37:59.927 INFO:teuthology.run_tasks:Running task install... 2024-05-08T05:37:59.936 DEBUG:teuthology.task.install:project ceph 2024-05-08T05:37:59.936 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'}} 2024-05-08T05:37:59.936 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'} 2024-05-08T05:37:59.936 INFO:teuthology.task.install:Using flavor: default 2024-05-08T05:37:59.944 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T05:37:59.944 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T05:37:59.944 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:37:59.945 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:37:59.947 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:37:59.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:37:59.948 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:37:59.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:38:00.094 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:38:00.095 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:38:00.098 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:38:00.098 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:38:00.103 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:38:00.103 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:38:00.213 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:38:00.213 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:38:00.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:38:00.239 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:38:00.239 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:38:00.271 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:38:00.271 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:38:00.272 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:38:00.276 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:38:00.276 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:38:00.276 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:38:00.295 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:38:00.295 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:38:00.303 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:38:00.305 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:38:00.305 DEBUG:teuthology.orchestra.run.smithi003:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:38:00.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:38:00.372 DEBUG:teuthology.orchestra.run.smithi003:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:38:00.376 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:38:00.434 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:38:00.439 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-05-08T05:38:00.441 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-05-08T05:38:00.449 DEBUG:teuthology.orchestra.run.smithi003:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:38:00.496 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-05-08T05:38:00.498 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-05-08T05:38:00.514 INFO:teuthology.orchestra.run.smithi003.stdout:check_obsoletes = 1 2024-05-08T05:38:00.516 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum clean all 2024-05-08T05:38:00.666 INFO:teuthology.orchestra.run.smithi089.stdout:54 files removed 2024-05-08T05:38:00.693 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:38:00.722 INFO:teuthology.orchestra.run.smithi104.stdout:54 files removed 2024-05-08T05:38:00.749 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:38:00.760 INFO:teuthology.orchestra.run.smithi003.stdout:54 files removed 2024-05-08T05:38:00.788 DEBUG:teuthology.orchestra.run.smithi003:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:38:01.393 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 190 kB/s | 84 kB 00:00 2024-05-08T05:38:01.565 INFO:teuthology.orchestra.run.smithi003.stdout:ceph packages for x86_64 172 kB/s | 84 kB 00:00 2024-05-08T05:38:01.761 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 112 kB/s | 84 kB 00:00 2024-05-08T05:38:01.902 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-05-08T05:38:02.416 INFO:teuthology.orchestra.run.smithi003.stdout:ceph noarch packages 15 kB/s | 12 kB 00:00 2024-05-08T05:38:02.419 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-05-08T05:38:02.628 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 14 kB/s | 12 kB 00:00 2024-05-08T05:38:03.003 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-08T05:38:04.049 INFO:teuthology.orchestra.run.smithi003.stdout:ceph source packages 1.2 kB/s | 1.9 kB 00:01 2024-05-08T05:38:04.137 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 1.3 kB/s | 1.9 kB 00:01 2024-05-08T05:38:06.649 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.1 MB 00:02 2024-05-08T05:38:06.970 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.1 MB 00:02 2024-05-08T05:38:07.195 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 19 MB 00:03 2024-05-08T05:38:08.567 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-08T05:38:11.096 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.2 MB 00:00 2024-05-08T05:38:12.955 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-08T05:38:14.710 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.2 MB 00:02 2024-05-08T05:38:15.051 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-08T05:38:16.152 INFO:teuthology.orchestra.run.smithi003.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-08T05:38:18.587 INFO:teuthology.orchestra.run.smithi003.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-05-08T05:38:20.132 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-05-08T05:38:22.218 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-05-08T05:38:22.416 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:22.417 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:22.427 INFO:teuthology.orchestra.run.smithi089.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:38:22.428 INFO:teuthology.orchestra.run.smithi089.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:38:22.429 INFO:teuthology.orchestra.run.smithi089.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:38:22.430 INFO:teuthology.orchestra.run.smithi089.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:38:22.432 INFO:teuthology.orchestra.run.smithi089.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:38:22.434 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:38:22.435 INFO:teuthology.orchestra.run.smithi089.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:38:22.495 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:38:22.512 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:38:22.513 INFO:teuthology.orchestra.run.smithi089.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:38:22.514 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:38:22.515 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:38:22.516 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:38:22.517 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:38:22.518 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:38:22.519 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:38:22.520 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:38:22.521 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:38:22.522 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:38:22.523 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-05-08T05:38:22.524 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-05-08T05:38:22.525 INFO:teuthology.orchestra.run.smithi089.stdout:Install 148 Packages 2024-05-08T05:38:22.525 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 8 Packages 2024-05-08T05:38:22.525 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:38:22.525 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 226 M 2024-05-08T05:38:22.525 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-05-08T05:38:23.142 INFO:teuthology.orchestra.run.smithi089.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 40 kB/s | 6.6 kB 00:00 2024-05-08T05:38:23.325 INFO:teuthology.orchestra.run.smithi089.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 4.7 MB/s | 890 kB 00:00 2024-05-08T05:38:23.376 INFO:teuthology.orchestra.run.smithi089.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 2.8 MB/s | 145 kB 00:00 2024-05-08T05:38:23.609 INFO:teuthology.orchestra.run.smithi089.stdout:(4/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 10 MB/s | 2.4 MB 00:00 2024-05-08T05:38:23.726 INFO:teuthology.orchestra.run.smithi089.stdout:(5/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 7.2 MB/s | 5.4 MB 00:00 2024-05-08T05:38:23.768 INFO:teuthology.orchestra.run.smithi089.stdout:(6/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 9.5 MB/s | 1.5 MB 00:00 2024-05-08T05:38:23.845 INFO:teuthology.orchestra.run.smithi003.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-08T05:38:24.219 INFO:teuthology.orchestra.run.smithi089.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 9.5 MB/s | 4.7 MB 00:00 2024-05-08T05:38:25.402 INFO:teuthology.orchestra.run.smithi089.stdout:(8/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x86 10 MB/s | 17 MB 00:01 2024-05-08T05:38:25.519 INFO:teuthology.orchestra.run.smithi089.stdout:(9/156): ceph-radosgw-19.0.0-3538.gbee299f4.el9 8.1 MB/s | 11 MB 00:01 2024-05-08T05:38:25.544 INFO:teuthology.orchestra.run.smithi089.stdout:(10/156): ceph-selinux-19.0.0-3538.gbee299f4.el 178 kB/s | 25 kB 00:00 2024-05-08T05:38:25.619 INFO:teuthology.orchestra.run.smithi089.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 2.9 MB/s | 225 kB 00:00 2024-05-08T05:38:25.695 INFO:teuthology.orchestra.run.smithi089.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 360 kB/s | 27 kB 00:00 2024-05-08T05:38:25.793 INFO:teuthology.orchestra.run.smithi089.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 323 kB/s | 32 kB 00:00 2024-05-08T05:38:26.012 INFO:teuthology.orchestra.run.smithi089.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 3.3 MB/s | 747 kB 00:00 2024-05-08T05:38:26.096 INFO:teuthology.orchestra.run.smithi089.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.9 MB/s | 163 kB 00:00 2024-05-08T05:38:26.179 INFO:teuthology.orchestra.run.smithi089.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.5 MB/s | 127 kB 00:00 2024-05-08T05:38:26.247 INFO:teuthology.orchestra.run.smithi003.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:26.248 INFO:teuthology.orchestra.run.smithi003.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:26.259 INFO:teuthology.orchestra.run.smithi003.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:38:26.260 INFO:teuthology.orchestra.run.smithi003.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:38:26.261 INFO:teuthology.orchestra.run.smithi003.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:38:26.262 INFO:teuthology.orchestra.run.smithi003.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:38:26.263 INFO:teuthology.orchestra.run.smithi003.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:38:26.266 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:38:26.267 INFO:teuthology.orchestra.run.smithi003.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:38:26.330 INFO:teuthology.orchestra.run.smithi089.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 3.3 MB/s | 500 kB 00:00 2024-05-08T05:38:26.332 INFO:teuthology.orchestra.run.smithi003.stdout:Dependencies resolved. 2024-05-08T05:38:26.360 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-08T05:38:26.360 INFO:teuthology.orchestra.run.smithi003.stdout: Package Arch Version Repository Size 2024-05-08T05:38:26.360 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-08T05:38:26.360 INFO:teuthology.orchestra.run.smithi003.stdout:Installing: 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:38:26.361 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:38:26.362 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrading: 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:38:26.363 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout:Installing dependencies: 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:38:26.364 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:38:26.365 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:38:26.366 INFO:teuthology.orchestra.run.smithi003.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:38:26.367 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:38:26.368 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:38:26.369 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:38:26.370 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:38:26.371 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout:Installing weak dependencies: 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction Summary 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout:======================================================================================= 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout:Install 148 Packages 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout:Upgrade 8 Packages 2024-05-08T05:38:26.372 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:38:26.373 INFO:teuthology.orchestra.run.smithi003.stdout:Total download size: 226 M 2024-05-08T05:38:26.373 INFO:teuthology.orchestra.run.smithi003.stdout:Downloading Packages: 2024-05-08T05:38:26.668 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-05-08T05:38:27.305 INFO:teuthology.orchestra.run.smithi089.stdout:(18/156): ceph-common-19.0.0-3538.gbee299f4.el9 4.9 MB/s | 21 MB 00:04 2024-05-08T05:38:27.313 INFO:teuthology.orchestra.run.smithi003.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-05-08T05:38:27.389 INFO:teuthology.orchestra.run.smithi089.stdout:(19/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 5.1 MB/s | 5.4 MB 00:01 2024-05-08T05:38:27.414 INFO:teuthology.orchestra.run.smithi089.stdout:(20/156): python3-ceph-argparse-19.0.0-3538.gbe 416 kB/s | 45 kB 00:00 2024-05-08T05:38:27.464 INFO:teuthology.orchestra.run.smithi089.stdout:(21/156): python3-ceph-common-19.0.0-3538.gbee2 1.8 MB/s | 135 kB 00:00 2024-05-08T05:38:27.548 INFO:teuthology.orchestra.run.smithi089.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 3.8 MB/s | 321 kB 00:00 2024-05-08T05:38:27.631 INFO:teuthology.orchestra.run.smithi089.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 3.5 MB/s | 299 kB 00:00 2024-05-08T05:38:27.707 INFO:teuthology.orchestra.run.smithi089.stdout:(24/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.3 MB/s | 100 kB 00:00 2024-05-08T05:38:27.790 INFO:teuthology.orchestra.run.smithi089.stdout:(25/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-08T05:38:27.832 INFO:teuthology.orchestra.run.smithi089.stdout:(26/156): python3-cephfs-19.0.0-3538.gbee299f4. 386 kB/s | 161 kB 00:00 2024-05-08T05:38:27.949 INFO:teuthology.orchestra.run.smithi089.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.4 MB/s | 171 kB 00:00 2024-05-08T05:38:28.066 INFO:teuthology.orchestra.run.smithi089.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 293 kB/s | 34 kB 00:00 2024-05-08T05:38:28.175 INFO:teuthology.orchestra.run.smithi089.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.3 MB/s | 142 kB 00:00 2024-05-08T05:38:28.197 INFO:teuthology.orchestra.run.smithi003.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 890 kB 00:00 2024-05-08T05:38:28.397 INFO:teuthology.orchestra.run.smithi003.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 724 kB/s | 145 kB 00:00 2024-05-08T05:38:28.502 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 16 kB 00:00 2024-05-08T05:38:28.900 INFO:teuthology.orchestra.run.smithi089.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 4.9 MB/s | 3.5 MB 00:00 2024-05-08T05:38:28.959 INFO:teuthology.orchestra.run.smithi089.stdout:(31/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 2.6 MB/s | 3.1 MB 00:01 2024-05-08T05:38:28.997 INFO:teuthology.orchestra.run.smithi003.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 2.7 MB/s | 5.4 MB 00:02 2024-05-08T05:38:29.134 INFO:teuthology.orchestra.run.smithi089.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.5 MB/s | 274 kB 00:00 2024-05-08T05:38:29.226 INFO:teuthology.orchestra.run.smithi089.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 540 kB/s | 49 kB 00:00 2024-05-08T05:38:29.318 INFO:teuthology.orchestra.run.smithi089.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 180 kB/s | 16 kB 00:00 2024-05-08T05:38:29.414 INFO:teuthology.orchestra.run.smithi003.stdout:(5/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-05-08T05:38:29.614 INFO:teuthology.orchestra.run.smithi003.stdout:(6/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-05-08T05:38:29.727 INFO:teuthology.orchestra.run.smithi089.stdout:(35/156): ceph-volume-19.0.0-3538.gbee299f4.el9 668 kB/s | 273 kB 00:00 2024-05-08T05:38:30.135 INFO:teuthology.orchestra.run.smithi089.stdout:(36/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 1.8 MB/s | 763 kB 00:00 2024-05-08T05:38:30.227 INFO:teuthology.orchestra.run.smithi089.stdout:(37/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 199 kB/s | 18 kB 00:00 2024-05-08T05:38:30.520 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-08T05:38:30.577 INFO:teuthology.orchestra.run.smithi089.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 230 kB/s | 80 kB 00:00 2024-05-08T05:38:30.614 INFO:teuthology.orchestra.run.smithi003.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-05-08T05:38:30.678 INFO:teuthology.orchestra.run.smithi089.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 4.2 MB/s | 7.4 MB 00:01 2024-05-08T05:38:31.086 INFO:teuthology.orchestra.run.smithi089.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-05-08T05:38:31.178 INFO:teuthology.orchestra.run.smithi089.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 390 kB/s | 195 kB 00:00 2024-05-08T05:38:31.204 INFO:teuthology.orchestra.run.smithi089.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-05-08T05:38:31.312 INFO:teuthology.orchestra.run.smithi089.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 984 kB/s | 106 kB 00:00 2024-05-08T05:38:31.421 INFO:teuthology.orchestra.run.smithi089.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-05-08T05:38:31.521 INFO:teuthology.orchestra.run.smithi089.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-05-08T05:38:31.630 INFO:teuthology.orchestra.run.smithi089.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-08T05:38:31.689 INFO:teuthology.orchestra.run.smithi089.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-05-08T05:38:31.747 INFO:teuthology.orchestra.run.smithi089.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-05-08T05:38:31.814 INFO:teuthology.orchestra.run.smithi089.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-05-08T05:38:31.848 INFO:teuthology.orchestra.run.smithi089.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-05-08T05:38:31.915 INFO:teuthology.orchestra.run.smithi089.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-08T05:38:31.965 INFO:teuthology.orchestra.run.smithi089.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-05-08T05:38:32.015 INFO:teuthology.orchestra.run.smithi089.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-05-08T05:38:32.116 INFO:teuthology.orchestra.run.smithi089.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-05-08T05:38:32.233 INFO:teuthology.orchestra.run.smithi089.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-05-08T05:38:32.291 INFO:teuthology.orchestra.run.smithi089.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.2 MB/s | 3.0 MB 00:00 2024-05-08T05:38:32.333 INFO:teuthology.orchestra.run.smithi089.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-05-08T05:38:32.356 INFO:teuthology.orchestra.run.smithi003.stdout:(8/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x86 6.1 MB/s | 17 MB 00:02 2024-05-08T05:38:32.392 INFO:teuthology.orchestra.run.smithi089.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-05-08T05:38:32.440 INFO:teuthology.orchestra.run.smithi003.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 303 kB/s | 25 kB 00:00 2024-05-08T05:38:32.442 INFO:teuthology.orchestra.run.smithi089.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-05-08T05:38:32.542 INFO:teuthology.orchestra.run.smithi089.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 255 kB/s | 25 kB 00:00 2024-05-08T05:38:32.643 INFO:teuthology.orchestra.run.smithi089.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-05-08T05:38:32.751 INFO:teuthology.orchestra.run.smithi089.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T05:38:32.860 INFO:teuthology.orchestra.run.smithi089.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-05-08T05:38:32.969 INFO:teuthology.orchestra.run.smithi089.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-05-08T05:38:33.277 INFO:teuthology.orchestra.run.smithi089.stdout:(65/156): libtool-2.4.6-45.el9.x86_64.rpm 658 kB/s | 582 kB 00:00 2024-05-08T05:38:33.507 INFO:teuthology.orchestra.run.smithi003.stdout:(10/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 3.7 MB/s | 11 MB 00:02 2024-05-08T05:38:33.632 INFO:teuthology.orchestra.run.smithi003.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 1.8 MB/s | 225 kB 00:00 2024-05-08T05:38:33.732 INFO:teuthology.orchestra.run.smithi003.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 270 kB/s | 27 kB 00:00 2024-05-08T05:38:33.833 INFO:teuthology.orchestra.run.smithi003.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 317 kB/s | 32 kB 00:00 2024-05-08T05:38:34.199 INFO:teuthology.orchestra.run.smithi003.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 2.0 MB/s | 747 kB 00:00 2024-05-08T05:38:34.308 INFO:teuthology.orchestra.run.smithi003.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.5 MB/s | 163 kB 00:00 2024-05-08T05:38:34.417 INFO:teuthology.orchestra.run.smithi003.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.1 MB/s | 127 kB 00:00 2024-05-08T05:38:34.692 INFO:teuthology.orchestra.run.smithi003.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 1.8 MB/s | 500 kB 00:00 2024-05-08T05:38:35.579 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-05-08T05:38:36.803 INFO:teuthology.orchestra.run.smithi003.stdout:(18/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 2.5 MB/s | 5.4 MB 00:02 2024-05-08T05:38:36.901 INFO:teuthology.orchestra.run.smithi003.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 451 kB/s | 45 kB 00:00 2024-05-08T05:38:36.911 INFO:teuthology.orchestra.run.smithi089.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 1.2 MB/s | 4.7 MB 00:03 2024-05-08T05:38:37.001 INFO:teuthology.orchestra.run.smithi003.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.3 MB/s | 135 kB 00:00 2024-05-08T05:38:37.093 INFO:teuthology.orchestra.run.smithi003.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.7 MB/s | 161 kB 00:00 2024-05-08T05:38:37.543 INFO:teuthology.orchestra.run.smithi003.stdout:(22/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 9.6 MB/s | 49 MB 00:05 2024-05-08T05:38:37.569 INFO:teuthology.orchestra.run.smithi003.stdout:(23/156): python3-rados-19.0.0-3538.gbee299f4.e 675 kB/s | 321 kB 00:00 2024-05-08T05:38:37.610 INFO:teuthology.orchestra.run.smithi003.stdout:(24/156): python3-rbd-19.0.0-3538.gbee299f4.el9 4.4 MB/s | 299 kB 00:00 2024-05-08T05:38:37.636 INFO:teuthology.orchestra.run.smithi003.stdout:(25/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.5 MB/s | 100 kB 00:00 2024-05-08T05:38:37.669 INFO:teuthology.orchestra.run.smithi003.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-08T05:38:37.728 INFO:teuthology.orchestra.run.smithi003.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 2.9 MB/s | 171 kB 00:00 2024-05-08T05:38:37.795 INFO:teuthology.orchestra.run.smithi003.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 514 kB/s | 34 kB 00:00 2024-05-08T05:38:37.853 INFO:teuthology.orchestra.run.smithi003.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 2.4 MB/s | 142 kB 00:00 2024-05-08T05:38:37.854 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:37.855 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:38:37.865 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:38:37.866 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:38:37.867 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:38:37.868 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:38:37.869 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:38:37.872 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:38:37.873 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:38:37.932 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-05-08T05:38:37.951 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-05-08T05:38:37.951 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-05-08T05:38:37.951 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-05-08T05:38:37.951 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-05-08T05:38:37.951 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:38:37.951 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:38:37.952 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:38:37.953 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:38:37.953 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:38:37.953 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:38:37.953 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:38:37.953 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:38:37.953 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:38:37.954 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:38:37.955 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-05-08T05:38:37.956 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:38:37.957 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:38:37.958 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:38:37.959 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:38:37.960 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:38:37.961 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:38:37.962 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:38:37.963 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:38:37.964 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:38:37.965 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:38:37.966 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout:Install 148 Packages 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 8 Packages 2024-05-08T05:38:37.967 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:38:37.968 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 226 M 2024-05-08T05:38:37.968 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-05-08T05:38:38.220 INFO:teuthology.orchestra.run.smithi003.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 9.7 MB/s | 3.5 MB 00:00 2024-05-08T05:38:38.244 INFO:teuthology.orchestra.run.smithi089.stdout:(67/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 3.9 MB/s | 49 MB 00:12 2024-05-08T05:38:38.371 INFO:teuthology.orchestra.run.smithi003.stdout:(31/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 4.2 MB/s | 3.1 MB 00:00 2024-05-08T05:38:38.529 INFO:teuthology.orchestra.run.smithi003.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.7 MB/s | 274 kB 00:00 2024-05-08T05:38:38.630 INFO:teuthology.orchestra.run.smithi003.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 494 kB/s | 49 kB 00:00 2024-05-08T05:38:38.730 INFO:teuthology.orchestra.run.smithi003.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 164 kB/s | 16 kB 00:00 2024-05-08T05:38:38.914 INFO:teuthology.orchestra.run.smithi003.stdout:(35/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.5 MB/s | 273 kB 00:00 2024-05-08T05:38:38.994 INFO:teuthology.orchestra.run.smithi089.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 1.2 MB/s | 911 kB 00:00 2024-05-08T05:38:39.014 INFO:teuthology.orchestra.run.smithi003.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 9.3 MB/s | 7.4 MB 00:00 2024-05-08T05:38:39.098 INFO:teuthology.orchestra.run.smithi003.stdout:(37/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 218 kB/s | 18 kB 00:00 2024-05-08T05:38:39.103 INFO:teuthology.orchestra.run.smithi089.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-05-08T05:38:39.187 INFO:teuthology.orchestra.run.smithi089.stdout:(70/156): postgresql-13.14-1.el9.x86_64.rpm 715 kB/s | 1.6 MB 00:02 2024-05-08T05:38:39.207 INFO:teuthology.orchestra.run.smithi003.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-05-08T05:38:39.313 INFO:teuthology.orchestra.run.smithi104.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 7.7 kB/s | 6.6 kB 00:00 2024-05-08T05:38:39.315 INFO:teuthology.orchestra.run.smithi003.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-05-08T05:38:39.329 INFO:teuthology.orchestra.run.smithi089.stdout:(71/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 749 kB/s | 4.4 MB 00:06 2024-05-08T05:38:39.365 INFO:teuthology.orchestra.run.smithi003.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-05-08T05:38:39.391 INFO:teuthology.orchestra.run.smithi003.stdout:(41/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 1.6 MB/s | 763 kB 00:00 2024-05-08T05:38:39.416 INFO:teuthology.orchestra.run.smithi003.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-05-08T05:38:39.466 INFO:teuthology.orchestra.run.smithi003.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T05:38:39.504 INFO:teuthology.orchestra.run.smithi089.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 500 kB/s | 87 kB 00:00 2024-05-08T05:38:39.516 INFO:teuthology.orchestra.run.smithi003.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-05-08T05:38:39.567 INFO:teuthology.orchestra.run.smithi003.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-08T05:38:39.617 INFO:teuthology.orchestra.run.smithi003.stdout:(46/156): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-05-08T05:38:39.642 INFO:teuthology.orchestra.run.smithi003.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-08T05:38:39.676 INFO:teuthology.orchestra.run.smithi003.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-05-08T05:38:39.710 INFO:teuthology.orchestra.run.smithi003.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-05-08T05:38:39.735 INFO:teuthology.orchestra.run.smithi003.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-05-08T05:38:39.760 INFO:teuthology.orchestra.run.smithi003.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-05-08T05:38:39.785 INFO:teuthology.orchestra.run.smithi003.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-05-08T05:38:39.811 INFO:teuthology.orchestra.run.smithi003.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-05-08T05:38:39.853 INFO:teuthology.orchestra.run.smithi003.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-05-08T05:38:39.895 INFO:teuthology.orchestra.run.smithi003.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-08T05:38:39.945 INFO:teuthology.orchestra.run.smithi003.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-05-08T05:38:40.020 INFO:teuthology.orchestra.run.smithi003.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-05-08T05:38:40.079 INFO:teuthology.orchestra.run.smithi003.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-05-08T05:38:40.137 INFO:teuthology.orchestra.run.smithi003.stdout:(59/156): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 582 kB 00:00 2024-05-08T05:38:40.180 INFO:teuthology.orchestra.run.smithi003.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 610 kB/s | 25 kB 00:00 2024-05-08T05:38:40.230 INFO:teuthology.orchestra.run.smithi003.stdout:(61/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.7 MB/s | 3.0 MB 00:00 2024-05-08T05:38:40.255 INFO:teuthology.orchestra.run.smithi003.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-05-08T05:38:40.280 INFO:teuthology.orchestra.run.smithi003.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-05-08T05:38:40.306 INFO:teuthology.orchestra.run.smithi003.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-05-08T05:38:40.331 INFO:teuthology.orchestra.run.smithi003.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 698 kB/s | 35 kB 00:00 2024-05-08T05:38:40.456 INFO:teuthology.orchestra.run.smithi003.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-05-08T05:38:40.523 INFO:teuthology.orchestra.run.smithi003.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-05-08T05:38:40.565 INFO:teuthology.orchestra.run.smithi003.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-05-08T05:38:40.579 INFO:teuthology.orchestra.run.smithi089.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 742 kB/s | 1.0 MB 00:01 2024-05-08T05:38:40.599 INFO:teuthology.orchestra.run.smithi003.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-05-08T05:38:40.624 INFO:teuthology.orchestra.run.smithi003.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-08T05:38:40.680 INFO:teuthology.orchestra.run.smithi104.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 652 kB/s | 890 kB 00:01 2024-05-08T05:38:40.700 INFO:teuthology.orchestra.run.smithi003.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-08T05:38:40.750 INFO:teuthology.orchestra.run.smithi003.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-08T05:38:40.825 INFO:teuthology.orchestra.run.smithi003.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-05-08T05:38:40.875 INFO:teuthology.orchestra.run.smithi003.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-08T05:38:40.921 INFO:teuthology.orchestra.run.smithi089.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 714 kB/s | 244 kB 00:00 2024-05-08T05:38:40.976 INFO:teuthology.orchestra.run.smithi003.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-05-08T05:38:40.980 INFO:teuthology.orchestra.run.smithi104.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 483 kB/s | 145 kB 00:00 2024-05-08T05:38:41.018 INFO:teuthology.orchestra.run.smithi003.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 621 kB/s | 26 kB 00:00 2024-05-08T05:38:41.060 INFO:teuthology.orchestra.run.smithi003.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-05-08T05:38:41.110 INFO:teuthology.orchestra.run.smithi003.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-08T05:38:41.152 INFO:teuthology.orchestra.run.smithi003.stdout:(79/156): python3-jinja2-2.11.3-5.el9.noarch.rp 897 kB/s | 248 kB 00:00 2024-05-08T05:38:41.163 INFO:teuthology.orchestra.run.smithi104.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-05-08T05:38:41.177 INFO:teuthology.orchestra.run.smithi003.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-05-08T05:38:41.228 INFO:teuthology.orchestra.run.smithi003.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-05-08T05:38:41.278 INFO:teuthology.orchestra.run.smithi003.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-05-08T05:38:41.288 INFO:teuthology.orchestra.run.smithi089.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 676 kB/s | 248 kB 00:00 2024-05-08T05:38:41.345 INFO:teuthology.orchestra.run.smithi003.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-05-08T05:38:41.370 INFO:teuthology.orchestra.run.smithi003.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 839 kB/s | 77 kB 00:00 2024-05-08T05:38:41.388 INFO:teuthology.orchestra.run.smithi089.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-05-08T05:38:41.395 INFO:teuthology.orchestra.run.smithi003.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-05-08T05:38:41.421 INFO:teuthology.orchestra.run.smithi003.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-05-08T05:38:41.446 INFO:teuthology.orchestra.run.smithi003.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-05-08T05:38:41.471 INFO:teuthology.orchestra.run.smithi003.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-05-08T05:38:41.488 INFO:teuthology.orchestra.run.smithi089.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-05-08T05:38:41.496 INFO:teuthology.orchestra.run.smithi003.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-08T05:38:41.521 INFO:teuthology.orchestra.run.smithi003.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-08T05:38:41.564 INFO:teuthology.orchestra.run.smithi003.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-08T05:38:41.605 INFO:teuthology.orchestra.run.smithi003.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-05-08T05:38:41.656 INFO:teuthology.orchestra.run.smithi003.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-08T05:38:41.698 INFO:teuthology.orchestra.run.smithi003.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 863 kB/s | 36 kB 00:00 2024-05-08T05:38:41.747 INFO:teuthology.orchestra.run.smithi089.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 666 kB/s | 172 kB 00:00 2024-05-08T05:38:41.848 INFO:teuthology.orchestra.run.smithi089.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-05-08T05:38:41.932 INFO:teuthology.orchestra.run.smithi003.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 203 kB/s | 47 kB 00:00 2024-05-08T05:38:41.982 INFO:teuthology.orchestra.run.smithi003.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-05-08T05:38:42.149 INFO:teuthology.orchestra.run.smithi003.stdout:(97/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-05-08T05:38:42.164 INFO:teuthology.orchestra.run.smithi104.stdout:(5/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 1.5 MB/s | 1.5 MB 00:00 2024-05-08T05:38:42.174 INFO:teuthology.orchestra.run.smithi003.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-05-08T05:38:42.191 INFO:teuthology.orchestra.run.smithi003.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-05-08T05:38:42.217 INFO:teuthology.orchestra.run.smithi003.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-08T05:38:42.242 INFO:teuthology.orchestra.run.smithi003.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-05-08T05:38:42.406 INFO:teuthology.orchestra.run.smithi089.stdout:(80/156): postgresql-server-13.14-1.el9.x86_64. 1.7 MB/s | 5.8 MB 00:03 2024-05-08T05:38:42.681 INFO:teuthology.orchestra.run.smithi104.stdout:(6/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 1.4 MB/s | 2.4 MB 00:01 2024-05-08T05:38:42.748 INFO:teuthology.orchestra.run.smithi089.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 734 kB/s | 250 kB 00:00 2024-05-08T05:38:43.025 INFO:teuthology.orchestra.run.smithi003.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 28 kB/s | 22 kB 00:00 2024-05-08T05:38:43.098 INFO:teuthology.orchestra.run.smithi089.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 635 kB/s | 222 kB 00:00 2024-05-08T05:38:43.282 INFO:teuthology.orchestra.run.smithi089.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-05-08T05:38:43.382 INFO:teuthology.orchestra.run.smithi089.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-05-08T05:38:43.642 INFO:teuthology.orchestra.run.smithi003.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 889 kB/s | 548 kB 00:00 2024-05-08T05:38:43.718 INFO:teuthology.orchestra.run.smithi003.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-05-08T05:38:43.756 INFO:teuthology.orchestra.run.smithi104.stdout:(7/156): ceph-common-19.0.0-3538.gbee299f4.el9. 4.0 MB/s | 21 MB 00:05 2024-05-08T05:38:43.784 INFO:teuthology.orchestra.run.smithi003.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-05-08T05:38:43.799 INFO:teuthology.orchestra.run.smithi089.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 646 kB/s | 269 kB 00:00 2024-05-08T05:38:43.826 INFO:teuthology.orchestra.run.smithi003.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 545 kB/s | 863 kB 00:01 2024-05-08T05:38:43.902 INFO:teuthology.orchestra.run.smithi003.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 350 kB/s | 26 kB 00:00 2024-05-08T05:38:44.066 INFO:teuthology.orchestra.run.smithi089.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 595 kB/s | 159 kB 00:00 2024-05-08T05:38:44.252 INFO:teuthology.orchestra.run.smithi003.stdout:(108/156): ceph-common-19.0.0-3538.gbee299f4.el 1.2 MB/s | 21 MB 00:17 2024-05-08T05:38:44.269 INFO:teuthology.orchestra.run.smithi003.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 133 kB/s | 49 kB 00:00 2024-05-08T05:38:44.344 INFO:teuthology.orchestra.run.smithi003.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.9 MB/s | 4.4 MB 00:00 2024-05-08T05:38:44.370 INFO:teuthology.orchestra.run.smithi003.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 578 kB/s | 67 kB 00:00 2024-05-08T05:38:44.395 INFO:teuthology.orchestra.run.smithi003.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-08T05:38:44.428 INFO:teuthology.orchestra.run.smithi003.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.9 MB/s | 839 kB 00:00 2024-05-08T05:38:44.470 INFO:teuthology.orchestra.run.smithi003.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 7.3 MB/s | 548 kB 00:00 2024-05-08T05:38:44.491 INFO:teuthology.orchestra.run.smithi089.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 656 kB/s | 279 kB 00:00 2024-05-08T05:38:44.495 INFO:teuthology.orchestra.run.smithi003.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 440 kB/s | 43 kB 00:00 2024-05-08T05:38:44.520 INFO:teuthology.orchestra.run.smithi003.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-05-08T05:38:44.546 INFO:teuthology.orchestra.run.smithi003.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-08T05:38:44.588 INFO:teuthology.orchestra.run.smithi003.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-08T05:38:44.592 INFO:teuthology.orchestra.run.smithi089.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-05-08T05:38:44.613 INFO:teuthology.orchestra.run.smithi003.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-05-08T05:38:44.638 INFO:teuthology.orchestra.run.smithi003.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-05-08T05:38:44.664 INFO:teuthology.orchestra.run.smithi003.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 477 kB/s | 35 kB 00:00 2024-05-08T05:38:44.681 INFO:teuthology.orchestra.run.smithi003.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-08T05:38:44.692 INFO:teuthology.orchestra.run.smithi089.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 536 kB/s | 54 kB 00:00 2024-05-08T05:38:44.773 INFO:teuthology.orchestra.run.smithi003.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-05-08T05:38:44.798 INFO:teuthology.orchestra.run.smithi003.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-08T05:38:44.823 INFO:teuthology.orchestra.run.smithi003.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 75 kB/s | 11 kB 00:00 2024-05-08T05:38:44.848 INFO:teuthology.orchestra.run.smithi003.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-08T05:38:44.882 INFO:teuthology.orchestra.run.smithi003.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-05-08T05:38:44.908 INFO:teuthology.orchestra.run.smithi003.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-05-08T05:38:44.933 INFO:teuthology.orchestra.run.smithi003.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-05-08T05:38:44.975 INFO:teuthology.orchestra.run.smithi003.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-05-08T05:38:45.000 INFO:teuthology.orchestra.run.smithi003.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-05-08T05:38:45.025 INFO:teuthology.orchestra.run.smithi003.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 869 kB/s | 79 kB 00:00 2024-05-08T05:38:45.050 INFO:teuthology.orchestra.run.smithi003.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 771 kB/s | 58 kB 00:00 2024-05-08T05:38:45.093 INFO:teuthology.orchestra.run.smithi003.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-08T05:38:45.118 INFO:teuthology.orchestra.run.smithi003.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-08T05:38:45.122 INFO:teuthology.orchestra.run.smithi104.stdout:(8/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 1.6 MB/s | 4.7 MB 00:02 2024-05-08T05:38:45.143 INFO:teuthology.orchestra.run.smithi003.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-05-08T05:38:45.168 INFO:teuthology.orchestra.run.smithi003.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-05-08T05:38:45.210 INFO:teuthology.orchestra.run.smithi003.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 461 kB/s | 31 kB 00:00 2024-05-08T05:38:45.222 INFO:teuthology.orchestra.run.smithi104.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 252 kB/s | 25 kB 00:00 2024-05-08T05:38:45.235 INFO:teuthology.orchestra.run.smithi003.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-08T05:38:45.260 INFO:teuthology.orchestra.run.smithi003.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-05-08T05:38:45.286 INFO:teuthology.orchestra.run.smithi003.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-05-08T05:38:45.328 INFO:teuthology.orchestra.run.smithi003.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 764 kB/s | 51 kB 00:00 2024-05-08T05:38:45.353 INFO:teuthology.orchestra.run.smithi003.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-05-08T05:38:45.378 INFO:teuthology.orchestra.run.smithi003.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 974 kB/s | 90 kB 00:00 2024-05-08T05:38:45.403 INFO:teuthology.orchestra.run.smithi003.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-08T05:38:45.445 INFO:teuthology.orchestra.run.smithi003.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 303 kB/s | 20 kB 00:00 2024-05-08T05:38:45.470 INFO:teuthology.orchestra.run.smithi003.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-05-08T05:38:45.521 INFO:teuthology.orchestra.run.smithi003.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-05-08T05:38:45.738 INFO:teuthology.orchestra.run.smithi003.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-05-08T05:38:45.805 INFO:teuthology.orchestra.run.smithi003.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-05-08T05:38:45.846 INFO:teuthology.orchestra.run.smithi003.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-08T05:38:45.897 INFO:teuthology.orchestra.run.smithi003.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-05-08T05:38:46.064 INFO:teuthology.orchestra.run.smithi003.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-05-08T05:38:46.114 INFO:teuthology.orchestra.run.smithi104.stdout:(10/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 4.5 MB/s | 11 MB 00:02 2024-05-08T05:38:46.156 INFO:teuthology.orchestra.run.smithi003.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-05-08T05:38:46.223 INFO:teuthology.orchestra.run.smithi104.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 2.0 MB/s | 225 kB 00:00 2024-05-08T05:38:46.306 INFO:teuthology.orchestra.run.smithi104.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 324 kB/s | 27 kB 00:00 2024-05-08T05:38:46.407 INFO:teuthology.orchestra.run.smithi104.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 317 kB/s | 32 kB 00:00 2024-05-08T05:38:46.682 INFO:teuthology.orchestra.run.smithi104.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 2.7 MB/s | 747 kB 00:00 2024-05-08T05:38:46.765 INFO:teuthology.orchestra.run.smithi104.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.9 MB/s | 163 kB 00:00 2024-05-08T05:38:46.789 INFO:teuthology.orchestra.run.smithi003.stdout:(155/156): librados2-19.0.0-3538.gbee299f4.el9. 2.5 MB/s | 3.4 MB 00:01 2024-05-08T05:38:46.858 INFO:teuthology.orchestra.run.smithi104.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.3 MB/s | 127 kB 00:00 2024-05-08T05:38:46.898 INFO:teuthology.orchestra.run.smithi003.stdout:(156/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 2.2 MB/s | 3.1 MB 00:01 2024-05-08T05:38:46.903 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:38:46.903 INFO:teuthology.orchestra.run.smithi003.stdout:Total 11 MB/s | 226 MB 00:20 2024-05-08T05:38:47.016 INFO:teuthology.orchestra.run.smithi104.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 3.1 MB/s | 500 kB 00:00 2024-05-08T05:38:47.817 INFO:teuthology.orchestra.run.smithi089.stdout:(90/156): python3-babel-2.9.1-2.el9.noarch.rpm 735 kB/s | 6.0 MB 00:08 2024-05-08T05:38:47.926 INFO:teuthology.orchestra.run.smithi089.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-05-08T05:38:48.034 INFO:teuthology.orchestra.run.smithi089.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-05-08T05:38:48.104 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction check 2024-05-08T05:38:48.143 INFO:teuthology.orchestra.run.smithi089.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-08T05:38:48.260 INFO:teuthology.orchestra.run.smithi089.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-05-08T05:38:48.271 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction check succeeded. 2024-05-08T05:38:48.271 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction test 2024-05-08T05:38:48.368 INFO:teuthology.orchestra.run.smithi089.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 438 kB/s | 47 kB 00:00 2024-05-08T05:38:48.399 INFO:teuthology.orchestra.run.smithi104.stdout:(18/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 3.9 MB/s | 5.4 MB 00:01 2024-05-08T05:38:48.477 INFO:teuthology.orchestra.run.smithi089.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 305 kB 00:00 2024-05-08T05:38:48.558 INFO:teuthology.orchestra.run.smithi104.stdout:(19/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 2.9 MB/s | 17 MB 00:05 2024-05-08T05:38:48.577 INFO:teuthology.orchestra.run.smithi089.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-05-08T05:38:48.644 INFO:teuthology.orchestra.run.smithi089.stdout:(98/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 768 kB/s | 5.1 MB 00:06 2024-05-08T05:38:48.678 INFO:teuthology.orchestra.run.smithi089.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 935 kB/s | 94 kB 00:00 2024-05-08T05:38:48.700 INFO:teuthology.orchestra.run.smithi104.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 959 kB/s | 135 kB 00:00 2024-05-08T05:38:48.759 INFO:teuthology.orchestra.run.smithi104.stdout:(21/156): python3-ceph-argparse-19.0.0-3538.gbe 126 kB/s | 45 kB 00:00 2024-05-08T05:38:48.779 INFO:teuthology.orchestra.run.smithi089.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-05-08T05:38:48.792 INFO:teuthology.orchestra.run.smithi104.stdout:(22/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.7 MB/s | 161 kB 00:00 2024-05-08T05:38:48.845 INFO:teuthology.orchestra.run.smithi089.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 317 kB/s | 64 kB 00:00 2024-05-08T05:38:48.879 INFO:teuthology.orchestra.run.smithi089.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-05-08T05:38:48.917 INFO:teuthology.orchestra.run.smithi104.stdout:(23/156): python3-rados-19.0.0-3538.gbee299f4.e 2.0 MB/s | 321 kB 00:00 2024-05-08T05:38:48.943 INFO:teuthology.orchestra.run.smithi104.stdout:(24/156): python3-rbd-19.0.0-3538.gbee299f4.el9 1.9 MB/s | 299 kB 00:00 2024-05-08T05:38:49.001 INFO:teuthology.orchestra.run.smithi104.stdout:(25/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.2 MB/s | 100 kB 00:00 2024-05-08T05:38:49.026 INFO:teuthology.orchestra.run.smithi104.stdout:(26/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-08T05:38:49.127 INFO:teuthology.orchestra.run.smithi104.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.7 MB/s | 171 kB 00:00 2024-05-08T05:38:49.163 INFO:teuthology.orchestra.run.smithi089.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-05-08T05:38:49.210 INFO:teuthology.orchestra.run.smithi104.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 410 kB/s | 34 kB 00:00 2024-05-08T05:38:49.229 INFO:teuthology.orchestra.run.smithi089.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-05-08T05:38:49.288 INFO:teuthology.orchestra.run.smithi089.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-05-08T05:38:49.336 INFO:teuthology.orchestra.run.smithi104.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.1 MB/s | 142 kB 00:00 2024-05-08T05:38:49.489 INFO:teuthology.orchestra.run.smithi089.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-08T05:38:49.539 INFO:teuthology.orchestra.run.smithi089.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-05-08T05:38:49.589 INFO:teuthology.orchestra.run.smithi089.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-05-08T05:38:49.647 INFO:teuthology.orchestra.run.smithi089.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-08T05:38:49.706 INFO:teuthology.orchestra.run.smithi089.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-08T05:38:49.770 INFO:teuthology.orchestra.run.smithi104.stdout:(30/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 4.0 MB/s | 3.1 MB 00:00 2024-05-08T05:38:49.782 INFO:teuthology.orchestra.run.smithi089.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-05-08T05:38:49.849 INFO:teuthology.orchestra.run.smithi089.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-05-08T05:38:49.899 INFO:teuthology.orchestra.run.smithi089.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 872 kB/s | 43 kB 00:00 2024-05-08T05:38:49.949 INFO:teuthology.orchestra.run.smithi089.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-05-08T05:38:50.016 INFO:teuthology.orchestra.run.smithi089.stdout:(115/156): protobuf-compiler-3.14.0-13.el9.x86_ 738 kB/s | 863 kB 00:01 2024-05-08T05:38:50.033 INFO:teuthology.orchestra.run.smithi089.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 179 kB/s | 15 kB 00:00 2024-05-08T05:38:50.075 INFO:teuthology.orchestra.run.smithi089.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-08T05:38:50.134 INFO:teuthology.orchestra.run.smithi089.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-05-08T05:38:50.192 INFO:teuthology.orchestra.run.smithi089.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-05-08T05:38:50.251 INFO:teuthology.orchestra.run.smithi089.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-05-08T05:38:50.360 INFO:teuthology.orchestra.run.smithi089.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-05-08T05:38:50.385 INFO:teuthology.orchestra.run.smithi089.stdout:(122/156): python3-cherrypy-18.6.1-2.el9.noarch 1.0 MB/s | 358 kB 00:00 2024-05-08T05:38:50.419 INFO:teuthology.orchestra.run.smithi089.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-08T05:38:50.444 INFO:teuthology.orchestra.run.smithi089.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 181 kB/s | 11 kB 00:00 2024-05-08T05:38:50.478 INFO:teuthology.orchestra.run.smithi089.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-05-08T05:38:50.503 INFO:teuthology.orchestra.run.smithi089.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-05-08T05:38:50.507 INFO:teuthology.orchestra.run.smithi003.stdout:Transaction test succeeded. 2024-05-08T05:38:50.507 INFO:teuthology.orchestra.run.smithi003.stdout:Running transaction 2024-05-08T05:38:50.536 INFO:teuthology.orchestra.run.smithi089.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-05-08T05:38:50.553 INFO:teuthology.orchestra.run.smithi089.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-05-08T05:38:50.612 INFO:teuthology.orchestra.run.smithi089.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-05-08T05:38:50.629 INFO:teuthology.orchestra.run.smithi089.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-05-08T05:38:50.644 INFO:teuthology.orchestra.run.smithi104.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 2.7 MB/s | 3.5 MB 00:01 2024-05-08T05:38:50.671 INFO:teuthology.orchestra.run.smithi089.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-08T05:38:50.688 INFO:teuthology.orchestra.run.smithi089.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 984 kB/s | 58 kB 00:00 2024-05-08T05:38:50.730 INFO:teuthology.orchestra.run.smithi089.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-05-08T05:38:50.747 INFO:teuthology.orchestra.run.smithi089.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-08T05:38:50.789 INFO:teuthology.orchestra.run.smithi089.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-08T05:38:50.806 INFO:teuthology.orchestra.run.smithi089.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-05-08T05:38:50.820 INFO:teuthology.orchestra.run.smithi104.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.5 MB/s | 274 kB 00:00 2024-05-08T05:38:50.848 INFO:teuthology.orchestra.run.smithi089.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 527 kB/s | 31 kB 00:00 2024-05-08T05:38:50.881 INFO:teuthology.orchestra.run.smithi089.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-08T05:38:50.907 INFO:teuthology.orchestra.run.smithi089.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-05-08T05:38:50.928 INFO:teuthology.orchestra.run.smithi104.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 456 kB/s | 49 kB 00:00 2024-05-08T05:38:50.940 INFO:teuthology.orchestra.run.smithi089.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 613 kB/s | 36 kB 00:00 2024-05-08T05:38:50.965 INFO:teuthology.orchestra.run.smithi089.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 874 kB/s | 51 kB 00:00 2024-05-08T05:38:51.016 INFO:teuthology.orchestra.run.smithi089.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-08T05:38:51.037 INFO:teuthology.orchestra.run.smithi104.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 152 kB/s | 16 kB 00:00 2024-05-08T05:38:51.041 INFO:teuthology.orchestra.run.smithi089.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-08T05:38:51.100 INFO:teuthology.orchestra.run.smithi089.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-05-08T05:38:51.125 INFO:teuthology.orchestra.run.smithi089.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-05-08T05:38:51.158 INFO:teuthology.orchestra.run.smithi089.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-08T05:38:51.229 INFO:teuthology.orchestra.run.smithi104.stdout:(35/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.4 MB/s | 273 kB 00:00 2024-05-08T05:38:51.275 INFO:teuthology.orchestra.run.smithi089.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-08T05:38:51.604 INFO:teuthology.orchestra.run.smithi104.stdout:(36/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 2.0 MB/s | 763 kB 00:00 2024-05-08T05:38:51.729 INFO:teuthology.orchestra.run.smithi104.stdout:(37/156): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-05-08T05:38:51.754 INFO:teuthology.orchestra.run.smithi104.stdout:(38/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 122 kB/s | 18 kB 00:00 2024-05-08T05:38:52.701 INFO:teuthology.orchestra.run.smithi089.stdout:(148/156): librados2-19.0.0-3538.gbee299f4.el9. 2.2 MB/s | 3.4 MB 00:01 2024-05-08T05:38:52.705 INFO:teuthology.orchestra.run.smithi104.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 82 kB/s | 80 kB 00:00 2024-05-08T05:38:52.851 INFO:teuthology.orchestra.run.smithi089.stdout:(149/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-05-08T05:38:53.080 INFO:teuthology.orchestra.run.smithi104.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 606 kB/s | 803 kB 00:01 2024-05-08T05:38:53.693 INFO:teuthology.orchestra.run.smithi089.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 2.2 MB/s | 1.8 MB 00:00 2024-05-08T05:38:53.735 INFO:teuthology.orchestra.run.smithi089.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-05-08T05:38:53.771 INFO:teuthology.orchestra.run.smithi104.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 183 kB/s | 195 kB 00:01 2024-05-08T05:38:53.802 INFO:teuthology.orchestra.run.smithi089.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 384 kB/s | 42 kB 00:00 2024-05-08T05:38:53.844 INFO:teuthology.orchestra.run.smithi089.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.2 MB/s | 248 kB 00:00 2024-05-08T05:38:53.928 INFO:teuthology.orchestra.run.smithi089.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 11 MB/s | 1.3 MB 00:00 2024-05-08T05:38:54.505 INFO:teuthology.orchestra.run.smithi104.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 178 kB/s | 253 kB 00:01 2024-05-08T05:38:54.770 INFO:teuthology.orchestra.run.smithi089.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.6 MB/s | 2.4 MB 00:00 2024-05-08T05:38:55.164 INFO:teuthology.orchestra.run.smithi104.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 902 kB/s | 1.2 MB 00:01 2024-05-08T05:38:55.472 INFO:teuthology.orchestra.run.smithi104.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 110 kB/s | 106 kB 00:00 2024-05-08T05:38:56.140 INFO:teuthology.orchestra.run.smithi104.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 139 kB/s | 135 kB 00:00 2024-05-08T05:38:56.882 INFO:teuthology.orchestra.run.smithi003.stdout: Preparing : 1/1 2024-05-08T05:38:56.889 INFO:teuthology.orchestra.run.smithi104.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 90 kB/s | 125 kB 00:01 2024-05-08T05:38:56.961 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:38:57.000 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:38:57.006 INFO:teuthology.orchestra.run.smithi104.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-05-08T05:38:57.055 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:38:57.095 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:38:57.098 INFO:teuthology.orchestra.run.smithi104.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 10 MB/s | 943 kB 00:00 2024-05-08T05:38:57.129 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:38:57.179 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:38:57.536 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:38:57.647 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:38:57.732 INFO:teuthology.orchestra.run.smithi104.stdout:(49/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 3.9 MB/s | 49 MB 00:12 2024-05-08T05:38:57.772 INFO:teuthology.orchestra.run.smithi104.stdout:(50/156): python3-urllib3-1.26.5-5.el9.noarch.r 132 kB/s | 215 kB 00:01 2024-05-08T05:38:57.775 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:38:57.807 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:38:57.807 INFO:teuthology.orchestra.run.smithi104.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 151 kB/s | 107 kB 00:00 2024-05-08T05:38:57.836 INFO:teuthology.orchestra.run.smithi104.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 214 kB/s | 21 kB 00:00 2024-05-08T05:38:57.875 INFO:teuthology.orchestra.run.smithi104.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-05-08T05:38:57.898 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:38:57.900 INFO:teuthology.orchestra.run.smithi104.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-05-08T05:38:57.936 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:38:57.942 INFO:teuthology.orchestra.run.smithi104.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-05-08T05:38:57.968 INFO:teuthology.orchestra.run.smithi104.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-05-08T05:38:58.027 INFO:teuthology.orchestra.run.smithi104.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-05-08T05:38:58.052 INFO:teuthology.orchestra.run.smithi104.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-08T05:38:58.101 INFO:teuthology.orchestra.run.smithi104.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-05-08T05:38:58.117 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:38:58.152 INFO:teuthology.orchestra.run.smithi104.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-05-08T05:38:58.175 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:38:58.176 INFO:teuthology.orchestra.run.smithi104.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-08T05:38:58.202 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:38:58.211 INFO:teuthology.orchestra.run.smithi104.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-05-08T05:38:58.235 INFO:teuthology.orchestra.run.smithi104.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-05-08T05:38:58.286 INFO:teuthology.orchestra.run.smithi104.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T05:38:58.307 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:38:58.311 INFO:teuthology.orchestra.run.smithi104.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-05-08T05:38:58.328 INFO:teuthology.orchestra.run.smithi104.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-05-08T05:38:58.335 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:38:58.408 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:38:58.437 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:38:58.459 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:38:58.486 INFO:teuthology.orchestra.run.smithi104.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-08T05:38:58.587 INFO:teuthology.orchestra.run.smithi104.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2024-05-08T05:38:58.614 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:38:58.654 INFO:teuthology.orchestra.run.smithi104.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-05-08T05:38:58.712 INFO:teuthology.orchestra.run.smithi104.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 3.9 MB/s | 911 kB 00:00 2024-05-08T05:38:58.746 INFO:teuthology.orchestra.run.smithi104.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 840 kB/s | 133 kB 00:00 2024-05-08T05:38:58.791 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:38:58.804 INFO:teuthology.orchestra.run.smithi104.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-08T05:38:58.880 INFO:teuthology.orchestra.run.smithi104.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-05-08T05:38:58.951 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:38:58.963 INFO:teuthology.orchestra.run.smithi104.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-08T05:38:58.994 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:38:59.047 INFO:teuthology.orchestra.run.smithi104.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-08T05:38:59.111 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:59.139 INFO:teuthology.orchestra.run.smithi104.stdout:(76/156): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-05-08T05:38:59.164 INFO:teuthology.orchestra.run.smithi104.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-08T05:38:59.179 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:59.206 INFO:teuthology.orchestra.run.smithi104.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-05-08T05:38:59.248 INFO:teuthology.orchestra.run.smithi104.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-05-08T05:38:59.274 INFO:teuthology.orchestra.run.smithi104.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 517 kB/s | 35 kB 00:00 2024-05-08T05:38:59.382 INFO:teuthology.orchestra.run.smithi104.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-05-08T05:38:59.402 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:38:59.433 INFO:teuthology.orchestra.run.smithi104.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-05-08T05:38:59.483 INFO:teuthology.orchestra.run.smithi104.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-08T05:38:59.508 INFO:teuthology.orchestra.run.smithi104.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-08T05:38:59.524 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:38:59.558 INFO:teuthology.orchestra.run.smithi104.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 564 kB/s | 42 kB 00:00 2024-05-08T05:38:59.559 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:38:59.594 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:38:59.609 INFO:teuthology.orchestra.run.smithi104.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-08T05:38:59.651 INFO:teuthology.orchestra.run.smithi104.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-08T05:38:59.679 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:38:59.718 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:38:59.734 INFO:teuthology.orchestra.run.smithi104.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 11 MB/s | 5.1 MB 00:00 2024-05-08T05:38:59.759 INFO:teuthology.orchestra.run.smithi104.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 471 kB/s | 51 kB 00:00 2024-05-08T05:38:59.762 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:38:59.785 INFO:teuthology.orchestra.run.smithi104.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-05-08T05:38:59.795 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:38:59.810 INFO:teuthology.orchestra.run.smithi104.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-05-08T05:38:59.821 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:38:59.860 INFO:teuthology.orchestra.run.smithi104.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-05-08T05:38:59.885 INFO:teuthology.orchestra.run.smithi104.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 586 kB/s | 44 kB 00:00 2024-05-08T05:38:59.886 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:38:59.936 INFO:teuthology.orchestra.run.smithi104.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-08T05:38:59.937 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:38:59.953 INFO:teuthology.orchestra.run.smithi104.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-05-08T05:38:59.966 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:39:00.003 INFO:teuthology.orchestra.run.smithi104.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-05-08T05:39:00.010 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:39:00.053 INFO:teuthology.orchestra.run.smithi104.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-08T05:39:00.078 INFO:teuthology.orchestra.run.smithi104.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-05-08T05:39:00.089 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:39:00.121 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:39:00.137 INFO:teuthology.orchestra.run.smithi104.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T05:39:00.162 INFO:teuthology.orchestra.run.smithi104.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 860 kB/s | 94 kB 00:00 2024-05-08T05:39:00.164 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:39:00.193 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:39:00.213 INFO:teuthology.orchestra.run.smithi104.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-05-08T05:39:00.246 INFO:teuthology.orchestra.run.smithi104.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-05-08T05:39:00.276 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:39:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:39:00.411 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:39:00.513 INFO:teuthology.orchestra.run.smithi104.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-08T05:39:00.555 INFO:teuthology.orchestra.run.smithi104.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-05-08T05:39:00.559 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:39:00.612 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:39:01.006 INFO:teuthology.orchestra.run.smithi104.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-05-08T05:39:01.176 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:39:01.259 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:39:01.293 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:39:01.341 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:39:01.374 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:39:01.379 INFO:teuthology.orchestra.run.smithi104.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 357 kB/s | 309 kB 00:00 2024-05-08T05:39:01.417 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:39:01.481 INFO:teuthology.orchestra.run.smithi104.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 257 kB/s | 26 kB 00:00 2024-05-08T05:39:01.514 INFO:teuthology.orchestra.run.smithi104.stdout:(108/156): abseil-cpp-20211102.0-3.el9.x86_64.r 432 kB/s | 548 kB 00:01 2024-05-08T05:39:01.590 INFO:teuthology.orchestra.run.smithi104.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 446 kB/s | 49 kB 00:00 2024-05-08T05:39:01.599 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:39:01.652 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:39:01.699 INFO:teuthology.orchestra.run.smithi104.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 366 kB/s | 67 kB 00:00 2024-05-08T05:39:01.748 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:39:01.810 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:39:01.850 INFO:teuthology.orchestra.run.smithi104.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 584 kB/s | 151 kB 00:00 2024-05-08T05:39:01.884 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:39:01.919 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:39:01.956 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:39:02.005 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:39:02.047 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:39:02.091 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:39:02.133 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:39:02.278 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:39:02.347 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:39:02.376 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:39:02.419 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:39:02.518 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:39:02.560 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:39:02.599 INFO:teuthology.orchestra.run.smithi104.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 731 kB/s | 548 kB 00:00 2024-05-08T05:39:02.601 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:39:02.634 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:39:02.658 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:39:02.708 INFO:teuthology.orchestra.run.smithi104.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 402 kB/s | 43 kB 00:00 2024-05-08T05:39:02.756 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:39:02.825 INFO:teuthology.orchestra.run.smithi104.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 745 kB/s | 839 kB 00:01 2024-05-08T05:39:02.850 INFO:teuthology.orchestra.run.smithi104.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 227 kB/s | 32 kB 00:00 2024-05-08T05:39:02.890 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:39:02.924 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:39:02.952 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:39:02.995 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:39:03.000 INFO:teuthology.orchestra.run.smithi104.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 100 kB/s | 15 kB 00:00 2024-05-08T05:39:03.034 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:39:03.074 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:39:03.103 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:39:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:39:03.171 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:39:03.213 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:39:03.217 INFO:teuthology.orchestra.run.smithi104.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 468 kB/s | 172 kB 00:00 2024-05-08T05:39:03.247 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:39:03.339 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:39:03.395 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:39:03.434 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:39:03.482 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:39:03.593 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:39:03.642 INFO:teuthology.orchestra.run.smithi104.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 558 kB/s | 358 kB 00:00 2024-05-08T05:39:03.645 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:39:03.751 INFO:teuthology.orchestra.run.smithi104.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-05-08T05:39:03.776 INFO:teuthology.orchestra.run.smithi104.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 542 kB/s | 303 kB 00:00 2024-05-08T05:39:04.093 INFO:teuthology.orchestra.run.smithi104.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 601 kB/s | 206 kB 00:00 2024-05-08T05:39:04.352 INFO:teuthology.orchestra.run.smithi104.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 560 kB/s | 144 kB 00:00 2024-05-08T05:39:04.418 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:39:04.460 INFO:teuthology.orchestra.run.smithi104.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2024-05-08T05:39:04.466 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:39:04.500 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:39:04.569 INFO:teuthology.orchestra.run.smithi104.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-05-08T05:39:04.605 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:39:04.667 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:39:04.677 INFO:teuthology.orchestra.run.smithi104.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2024-05-08T05:39:04.759 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:39:04.778 INFO:teuthology.orchestra.run.smithi104.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-05-08T05:39:04.793 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:39:04.878 INFO:teuthology.orchestra.run.smithi104.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-05-08T05:39:04.977 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:39:05.016 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:39:05.097 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:39:05.793 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:39:05.902 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:39:06.186 INFO:teuthology.orchestra.run.smithi104.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 799 kB/s | 1.0 MB 00:01 2024-05-08T05:39:06.286 INFO:teuthology.orchestra.run.smithi089.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 771 kB/s | 16 MB 00:21 2024-05-08T05:39:06.289 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:39:06.289 INFO:teuthology.orchestra.run.smithi089.stdout:Total 5.2 MB/s | 226 MB 00:43 2024-05-08T05:39:06.295 INFO:teuthology.orchestra.run.smithi104.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-05-08T05:39:06.387 INFO:teuthology.orchestra.run.smithi104.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 799 kB/s | 2.0 MB 00:02 2024-05-08T05:39:06.487 INFO:teuthology.orchestra.run.smithi104.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2024-05-08T05:39:06.596 INFO:teuthology.orchestra.run.smithi104.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 277 kB/s | 58 kB 00:00 2024-05-08T05:39:06.696 INFO:teuthology.orchestra.run.smithi104.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-05-08T05:39:06.880 INFO:teuthology.orchestra.run.smithi104.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 490 kB/s | 90 kB 00:00 2024-05-08T05:39:06.913 INFO:teuthology.orchestra.run.smithi104.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 640 kB/s | 272 kB 00:00 2024-05-08T05:39:06.981 INFO:teuthology.orchestra.run.smithi104.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 282 kB/s | 28 kB 00:00 2024-05-08T05:39:07.006 INFO:teuthology.orchestra.run.smithi104.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 333 kB/s | 31 kB 00:00 2024-05-08T05:39:07.123 INFO:teuthology.orchestra.run.smithi104.stdout:(138/156): libarrow-9.0.0-11.el9.x86_64.rpm 739 kB/s | 4.4 MB 00:06 2024-05-08T05:39:07.223 INFO:teuthology.orchestra.run.smithi104.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 273 kB/s | 59 kB 00:00 2024-05-08T05:39:07.237 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:39:07.240 INFO:teuthology.orchestra.run.smithi104.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 306 kB/s | 36 kB 00:00 2024-05-08T05:39:07.265 INFO:teuthology.orchestra.run.smithi104.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 662 kB/s | 188 kB 00:00 2024-05-08T05:39:07.312 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:39:07.315 INFO:teuthology.orchestra.run.smithi104.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-05-08T05:39:07.340 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:39:07.365 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:39:07.434 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-05-08T05:39:07.449 INFO:teuthology.orchestra.run.smithi104.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 489 kB/s | 90 kB 00:00 2024-05-08T05:39:07.557 INFO:teuthology.orchestra.run.smithi104.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 184 kB/s | 20 kB 00:00 2024-05-08T05:39:07.583 INFO:teuthology.orchestra.run.smithi104.stdout:(145/156): python3-webob-1.8.7-6.el9.noarch.rpm 670 kB/s | 230 kB 00:00 2024-05-08T05:39:07.586 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-05-08T05:39:07.586 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-05-08T05:39:07.694 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:39:07.735 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:07.810 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:07.833 INFO:teuthology.orchestra.run.smithi104.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 696 kB/s | 192 kB 00:00 2024-05-08T05:39:07.840 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:39:07.877 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:39:07.908 INFO:teuthology.orchestra.run.smithi104.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 721 kB/s | 427 kB 00:00 2024-05-08T05:39:08.367 INFO:teuthology.orchestra.run.smithi104.stdout:(148/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 6.9 MB/s | 3.1 MB 00:00 2024-05-08T05:39:08.397 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:39:08.422 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:08.493 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:08.515 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:39:08.575 INFO:teuthology.orchestra.run.smithi104.stdout:(149/156): librados2-19.0.0-3538.gbee299f4.el9. 4.6 MB/s | 3.4 MB 00:00 2024-05-08T05:39:09.460 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-05-08T05:39:09.461 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-05-08T05:39:09.651 INFO:teuthology.orchestra.run.smithi104.stdout:(150/156): thrift-0.15.0-2.el9.x86_64.rpm 783 kB/s | 1.6 MB 00:02 2024-05-08T05:39:09.843 INFO:teuthology.orchestra.run.smithi104.stdout:(151/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.6 MB/s | 2.3 MB 00:01 2024-05-08T05:39:10.026 INFO:teuthology.orchestra.run.smithi104.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-05-08T05:39:10.526 INFO:teuthology.orchestra.run.smithi104.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 48 kB/s | 42 kB 00:00 2024-05-08T05:39:10.972 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:10.993 INFO:teuthology.orchestra.run.smithi104.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 216 kB/s | 248 kB 00:01 2024-05-08T05:39:11.003 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:11.443 INFO:teuthology.orchestra.run.smithi104.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 975 kB/s | 1.3 MB 00:01 2024-05-08T05:39:11.586 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:11.623 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:11.717 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:11.717 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:39:11.718 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:39:11.718 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:11.758 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:39:12.418 INFO:teuthology.orchestra.run.smithi104.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 2.4 MB 00:01 2024-05-08T05:39:12.426 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:39:12.426 INFO:teuthology.orchestra.run.smithi104.stdout:Total 6.6 MB/s | 226 MB 00:34 2024-05-08T05:39:13.594 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-05-08T05:39:13.747 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-05-08T05:39:13.747 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-05-08T05:39:15.535 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-05-08T05:39:15.538 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-05-08T05:39:15.539 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-05-08T05:39:15.614 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:39:15.661 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:39:15.714 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:39:15.748 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:39:15.774 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:39:15.816 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:39:16.195 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:39:16.250 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:16.373 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:16.409 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:16.518 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:16.547 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:39:16.694 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:39:16.744 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:39:16.772 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:16.875 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:16.898 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:39:16.960 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:39:16.982 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:17.007 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:17.199 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:17.326 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:17.471 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:17.532 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:17.650 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:17.717 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:17.904 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:18.012 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:39:18.037 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:39:18.064 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:39:18.128 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:39:18.163 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:39:18.207 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:39:18.240 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:39:18.267 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:39:18.336 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:39:18.391 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:39:18.420 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:39:18.456 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:39:18.524 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:39:18.559 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:39:18.601 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:39:18.630 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:39:18.699 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:39:18.745 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:39:18.819 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:39:18.968 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:39:19.022 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:39:19.552 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:39:19.627 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:39:19.664 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:39:19.699 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:39:19.737 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:39:19.778 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:39:19.924 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:39:19.990 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:39:20.081 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:39:20.145 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:39:20.218 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:39:20.250 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:39:20.286 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:39:20.333 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:39:20.369 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:39:20.412 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:39:20.454 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:39:20.589 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:39:20.635 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:39:20.665 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:39:20.706 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:39:20.801 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:39:20.835 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:39:20.872 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:39:20.897 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:39:20.922 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:39:20.995 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:39:21.123 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:39:21.172 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:39:21.199 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:39:21.234 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:39:21.273 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:39:21.304 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:39:21.334 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:39:21.369 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:39:21.402 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:39:21.444 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:39:21.478 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:39:21.564 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-05-08T05:39:21.565 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:39:21.608 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:39:21.640 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:39:21.663 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:39:21.687 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:39:21.702 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:39:21.755 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:39:21.785 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:39:21.789 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:39:21.814 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:39:21.841 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:39:21.857 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:39:22.176 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:39:22.332 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:22.454 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:22.483 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:22.554 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:39:22.573 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:22.604 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:39:22.613 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:39:22.647 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:39:22.745 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:39:22.755 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:39:22.797 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:39:22.800 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:39:22.829 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:22.884 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:39:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:22.925 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:39:22.938 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:39:22.984 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:39:23.005 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:23.027 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:39:23.101 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:39:23.131 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:39:23.207 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:39:23.222 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:23.387 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:39:23.527 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:23.588 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:39:23.704 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:23.847 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:39:23.950 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:39:23.953 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:39:24.061 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:39:24.093 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:39:24.128 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:39:24.140 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:39:24.140 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /sys 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /proc 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /mnt 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /var/tmp 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /home 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /root 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout:skipping the directory /tmp 2024-05-08T05:39:24.141 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:24.193 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:39:24.227 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:39:24.270 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:39:24.303 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:39:24.324 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:39:24.331 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:39:24.343 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:24.391 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:39:24.445 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:39:24.476 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:39:24.511 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:39:24.587 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:39:24.622 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:39:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:39:24.685 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:39:24.756 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:39:24.808 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:39:24.883 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:39:25.018 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:39:25.069 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:39:25.170 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:39:25.243 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:39:25.271 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:39:25.304 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:39:25.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:39:25.617 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:39:25.659 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:25.698 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:39:25.726 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:39:25.766 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:25.769 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:39:25.797 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:39:25.807 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:39:25.833 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:39:25.849 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:39:25.877 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:25.906 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:25.906 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:25.906 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:39:25.906 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:25.906 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:25.906 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:26.013 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:39:26.085 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:39:26.136 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:26.161 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:26.161 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:26.161 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:39:26.161 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:26.161 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:26.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:26.176 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:39:26.266 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:39:26.313 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:39:26.339 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:39:26.345 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:26.369 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:39:26.406 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:39:26.416 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:26.439 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:39:26.453 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:39:26.488 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:39:26.532 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:39:26.574 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:39:26.620 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:26.645 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:39:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:26.645 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:26.645 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:26.710 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:39:26.754 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:39:26.759 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:39:26.787 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:39:26.818 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:39:26.825 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:39:26.882 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:39:26.920 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:39:26.959 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:39:26.999 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:39:27.024 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:39:27.049 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:39:27.079 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:39:27.124 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:39:27.138 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:39:27.211 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:39:27.251 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:39:27.289 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:39:27.310 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:39:27.317 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:39:27.382 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:39:27.416 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:39:27.447 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:39:27.453 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:39:27.477 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:39:27.511 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:39:27.513 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:39:27.545 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:39:27.548 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:39:27.580 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:39:27.586 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:39:27.620 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:39:27.707 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:39:27.751 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:39:27.790 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:39:27.837 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:39:27.936 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:39:27.983 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:39:28.080 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:28.105 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:28.138 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:39:28.203 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:39:28.227 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:28.682 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:28.714 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:39:28.763 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:39:28.797 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:39:28.803 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:28.898 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:39:28.955 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:39:29.043 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:39:29.115 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:39:29.291 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:39:29.327 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:29.329 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:39:29.348 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:29.362 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:29.390 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:29.407 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:39:29.480 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:39:29.480 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:39:29.480 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:29.508 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:39:30.062 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:39:30.170 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:39:30.383 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:30.413 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:30.532 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:30.698 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:39:30.732 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:30.757 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:30.757 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:30.757 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:39:30.757 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:30.757 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:30.757 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:30.808 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:30.819 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:30.921 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:39:30.982 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:39:31.404 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:39:31.499 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:39:31.527 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:39:31.560 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:39:31.873 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:39:31.915 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:31.963 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:39:31.993 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:32.004 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:39:32.018 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:32.018 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:32.018 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:39:32.019 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:32.019 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:32.019 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:32.036 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:39:32.060 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:32.072 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:39:32.084 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:32.085 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:32.085 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:39:32.085 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:32.136 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:32.161 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:32.161 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:32.161 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:39:32.161 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:32.161 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:32.161 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:32.478 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:32.503 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:32.503 INFO:teuthology.orchestra.run.smithi003.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:32.503 INFO:teuthology.orchestra.run.smithi003.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:39:32.503 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:32.504 INFO:teuthology.orchestra.run.smithi003.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:32.504 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:32.549 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:39:32.575 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:32.663 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:39:32.685 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:39:34.953 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:34.990 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:35.531 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:39:35.569 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:35.645 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:39:35.645 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:39:35.645 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:39:35.645 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:35.670 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:39:37.383 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:39:37.443 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:39:37.482 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:39:37.513 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:39:37.540 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:39:37.756 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:39:37.812 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:39:37.842 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:39:37.872 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:39:37.918 INFO:teuthology.orchestra.run.smithi003.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:39:38.524 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:38.565 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:38.587 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:39.500 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:39:39.587 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:39:39.633 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:39:39.733 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:39:39.765 INFO:teuthology.orchestra.run.smithi003.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:39:39.791 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:39:39.791 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:39.876 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:39.908 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:39:39.909 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:40.001 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:40.026 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:39:40.032 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:40.032 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:40.220 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:40.220 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:40.304 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:40.372 INFO:teuthology.orchestra.run.smithi003.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:41.098 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:39:41.098 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-05-08T05:39:41.098 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-05-08T05:39:41.099 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-05-08T05:39:41.099 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-05-08T05:39:41.099 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-05-08T05:39:41.099 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-05-08T05:39:41.099 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-05-08T05:39:41.099 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:41.264 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:39:41.279 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:39:41.279 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:41.279 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:39:41.279 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:42.725 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:42.749 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:42.749 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:42.749 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:39:42.749 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:42.749 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:42.749 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:42.974 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:42.996 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:42.996 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:42.996 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:39:42.996 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:42.996 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:42.997 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:43.433 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:43.455 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:43.455 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:43.455 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:39:43.455 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:43.455 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:43.455 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:43.563 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:39:43.612 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:39:43.667 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:39:43.853 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:39:43.901 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:39:43.978 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:39:44.066 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:39:44.226 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:39:44.284 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:39:44.316 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:39:44.344 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:39:44.770 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:44.801 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:44.824 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:39:44.885 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:39:44.915 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:45.951 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:45.986 INFO:teuthology.orchestra.run.smithi003.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:45.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:39:45.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:39:45.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:39:45.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:39:45.986 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:39:45.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:39:45.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:39:45.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:39:45.989 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:39:45.990 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:39:45.991 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:39:45.992 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:39:45.993 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:39:45.994 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:39:45.995 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:39:45.996 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:39:45.997 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:39:45.998 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:39:45.999 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:39:46.000 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:39:46.001 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:39:46.002 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:46.951 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:46.984 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:47.094 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:47.255 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:39:47.278 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:47.300 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:47.300 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:47.300 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:39:47.300 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:47.300 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:47.300 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-05-08T05:39:47.326 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-05-08T05:39:47.327 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:47.345 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:47.356 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:47.454 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:39:47.492 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:39:47.501 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:39:47.507 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:39:47.507 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:47.507 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:39:47.507 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:48.431 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:39:48.464 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:48.487 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:48.487 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:48.487 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:39:48.487 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:48.487 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:48.487 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:48.532 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:48.553 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:48.553 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:48.553 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:39:48.553 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:48.599 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout:Upgraded: 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:48.618 INFO:teuthology.orchestra.run.smithi003.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout:Installed: 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.619 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.620 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:39:48.621 INFO:teuthology.orchestra.run.smithi003.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.622 INFO:teuthology.orchestra.run.smithi003.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:39:48.623 INFO:teuthology.orchestra.run.smithi003.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:39:48.624 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:39:48.625 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:39:48.626 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:39:48.627 INFO:teuthology.orchestra.run.smithi003.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:39:48.628 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:39:48.629 INFO:teuthology.orchestra.run.smithi003.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:39:48.630 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:39:48.630 INFO:teuthology.orchestra.run.smithi003.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:39:48.630 INFO:teuthology.orchestra.run.smithi003.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:39:48.630 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.630 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi003.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:39:48.631 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi003.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:39:48.632 INFO:teuthology.orchestra.run.smithi003.stdout:Complete! 2024-05-08T05:39:48.925 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:48.947 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:48.947 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:48.947 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:39:48.947 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:48.948 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:48.948 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:39:48.990 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:49.016 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:39:49.016 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:49.016 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:39:49.016 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:49.016 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:39:49.016 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:49.073 DEBUG:teuthology.parallel:result is None 2024-05-08T05:39:49.246 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:49.268 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:39:49.268 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:49.269 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:39:49.269 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:49.269 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:39:49.269 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:49.721 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:49.743 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:39:49.744 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:49.744 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:39:49.744 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:49.744 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:39:49.744 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:49.844 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:39:49.892 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:39:49.947 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:39:50.133 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:39:50.172 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:39:50.250 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:39:50.346 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:39:50.489 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:39:50.538 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:39:50.570 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:39:50.598 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:39:51.042 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:51.073 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:39:51.103 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:39:51.165 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:39:51.194 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:52.227 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:39:52.271 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:53.236 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:39:53.263 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:53.373 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:39:53.542 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:39:53.545 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:39:53.573 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:53.586 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:39:53.596 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:39:53.596 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:53.596 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:39:53.596 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:53.596 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:39:53.597 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:53.626 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:39:53.641 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:53.651 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:39:53.659 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:39:53.686 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:39:53.749 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:39:53.797 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:39:53.866 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:39:53.908 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:39:53.938 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:39:53.967 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:39:54.005 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:39:54.630 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:54.670 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:54.689 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:39:54.715 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:39:54.743 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:54.765 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:39:54.765 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:54.766 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:39:54.766 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:54.766 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:39:54.766 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:54.810 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:54.832 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:39:54.832 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:54.832 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:39:54.833 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:54.877 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:54.900 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:39:54.900 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:54.900 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:39:54.900 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:54.900 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:39:54.900 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:55.203 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:55.226 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:39:55.226 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:39:55.226 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:39:55.226 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:55.226 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:39:55.226 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:39:55.508 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:39:55.599 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:39:55.641 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:39:55.723 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:39:55.753 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:39:55.779 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:39:55.779 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:55.855 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:39:55.888 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:39:55.888 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:55.955 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:39:55.980 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:56.165 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:39:56.166 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:39:56.322 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:39:59.815 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:39:59.856 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:39:59.897 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:39:59.929 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:39:59.956 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:40:00.135 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:40:00.177 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:40:00.208 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:40:00.237 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:40:00.292 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:40:00.923 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:00.965 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:00.985 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:40:01.824 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:40:01.876 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:01.876 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:40:01.876 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:40:01.876 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:40:01.877 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:40:01.878 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:40:01.879 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:40:01.879 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:40:01.879 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:40:01.891 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:40:01.891 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:40:01.891 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:40:01.891 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:40:01.892 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:40:01.893 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:40:01.894 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:40:01.895 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:40:01.895 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:40:01.895 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:40:01.895 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:40:01.895 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:40:01.895 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:40:01.896 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:40:01.897 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:40:01.897 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:40:01.897 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:40:01.897 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:40:01.898 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:40:01.899 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:40:01.900 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:40:01.901 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:40:01.902 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:40:01.903 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:40:01.903 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:40:01.911 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:40:01.944 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:40:02.036 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:40:02.072 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:40:02.098 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:40:02.098 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:02.183 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:40:02.216 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:40:02.216 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:02.300 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:40:02.325 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:40:02.330 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:02.330 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:02.510 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:40:02.510 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:02.586 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:40:02.650 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:04.608 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:04.609 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:40:04.610 INFO:teuthology.orchestra.run.smithi089.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.611 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.612 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.613 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:40:04.614 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:40:04.615 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:40:04.616 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:40:04.617 INFO:teuthology.orchestra.run.smithi089.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:40:04.618 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:40:04.619 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.620 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:40:04.621 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:40:04.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:40:04.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:40:04.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:40:04.625 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:40:04.626 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:40:04.627 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:40:04.628 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:40:04.629 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi089.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi089.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:40:04.630 INFO:teuthology.orchestra.run.smithi089.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:40:04.631 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-05-08T05:40:04.631 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-05-08T05:40:05.074 DEBUG:teuthology.parallel:result is None 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:40:08.150 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:40:08.151 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:40:08.152 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:40:08.153 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:40:08.154 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:40:08.155 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:40:08.156 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:40:08.157 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:40:08.158 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:40:08.159 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:40:08.160 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:40:08.161 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:40:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:40:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:40:08.162 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:40:10.618 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:40:10.618 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:40:10.619 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.620 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:40:10.621 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:40:10.622 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:40:10.623 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:40:10.624 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:40:10.625 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:40:10.626 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:40:10.627 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:40:10.628 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.629 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:40:10.630 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-05-08T05:40:11.066 DEBUG:teuthology.parallel:result is None 2024-05-08T05:40:11.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:11.216 DEBUG:teuthology.orchestra.run.smithi003:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:11.244 INFO:teuthology.orchestra.run.smithi003.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:11.244 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:11.244 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:40:11.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:11.399 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:11.423 INFO:teuthology.orchestra.run.smithi089.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:11.423 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:11.423 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:40:11.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:11.568 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:40:11.592 INFO:teuthology.orchestra.run.smithi104.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:11.592 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:40:11.592 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:40:11.594 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T05:40:11.595 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:40:11.595 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:11.638 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:40:11.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:11.667 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:40:11.668 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:40:11.695 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T05:40:11.695 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:40:11.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:11.725 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:11.795 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:40:11.795 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:11.821 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:11.886 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:40:11.887 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:40:11.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:40:11.979 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T05:40:11.980 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:40:11.980 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:12.008 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:12.074 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:40:12.074 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:12.098 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:12.161 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:40:12.161 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:40:12.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:40:12.251 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T05:40:12.251 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:40:12.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:12.278 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:12.344 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:40:12.344 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:12.367 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:12.430 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:40:12.430 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:40:12.456 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:40:12.521 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T05:40:12.622 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-08T05:40:12.622 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:12.623 INFO:tasks.cephadm:Cluster fsid is 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:40:12.623 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T05:40:12.623 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.3', 'mon.b': '172.21.15.89', 'mon.c': '172.21.15.104'} 2024-05-08T05:40:12.623 INFO:tasks.cephadm:First mon is mon.a on smithi003 2024-05-08T05:40:12.623 INFO:tasks.cephadm:First mgr is x 2024-05-08T05:40:12.623 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T05:40:12.623 DEBUG:teuthology.orchestra.run.smithi003:> sudo hostname $(hostname -s) 2024-05-08T05:40:12.650 DEBUG:teuthology.orchestra.run.smithi089:> sudo hostname $(hostname -s) 2024-05-08T05:40:12.673 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-05-08T05:40:12.699 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T05:40:12.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:12.927 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79166/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3538-gbee299f4', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3538.gbee299f4'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 17:50:48.409840', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-57676-3', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T05:40:13.072 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-57676-3, sha1 bee299f4ac344e4c45e550d7a80abcadd7bc3d5a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:40:13.074 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm 2024-05-08T05:40:13.074 DEBUG:teuthology.orchestra.run.smithi003:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:14.202 INFO:teuthology.orchestra.run.smithi003.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:40 /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:14.203 DEBUG:teuthology.orchestra.run.smithi089:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:14.940 INFO:teuthology.orchestra.run.smithi089.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:40 /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:14.940 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:15.806 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:40 /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:15.806 DEBUG:teuthology.orchestra.run.smithi003:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:15.826 DEBUG:teuthology.orchestra.run.smithi089:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:15.846 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:40:15.880 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a on all hosts... 2024-05-08T05:40:15.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:40:15.883 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:40:15.889 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:40:17.348 INFO:teuthology.orchestra.run.smithi003.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:40:17.452 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:40:17.636 INFO:teuthology.orchestra.run.smithi089.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:41:18.456 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-08T05:41:18.456 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:41:18.456 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:41:18.457 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-05-08T05:41:18.457 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:41:18.457 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-05-08T05:41:18.457 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-08T05:41:18.931 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-05-08T05:41:18.931 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:41:18.932 INFO:teuthology.orchestra.run.smithi089.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:41:18.932 INFO:teuthology.orchestra.run.smithi089.stdout: "repo_digests": [ 2024-05-08T05:41:18.932 INFO:teuthology.orchestra.run.smithi089.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:41:18.932 INFO:teuthology.orchestra.run.smithi089.stdout: ] 2024-05-08T05:41:18.932 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-05-08T05:41:24.880 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:41:24.880 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:41:24.880 INFO:teuthology.orchestra.run.smithi003.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:41:24.880 INFO:teuthology.orchestra.run.smithi003.stdout: "repo_digests": [ 2024-05-08T05:41:24.880 INFO:teuthology.orchestra.run.smithi003.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:41:24.881 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T05:41:24.881 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:41:24.906 DEBUG:teuthology.orchestra.run.smithi003:> sudo mkdir -p /etc/ceph 2024-05-08T05:41:24.956 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph 2024-05-08T05:41:24.988 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-05-08T05:41:25.021 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 777 /etc/ceph 2024-05-08T05:41:25.048 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /etc/ceph 2024-05-08T05:41:25.074 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-05-08T05:41:25.103 INFO:tasks.cephadm:Writing seed config... 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T05:41:25.104 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T05:41:25.105 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T05:41:25.106 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T05:41:25.107 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T05:41:25.107 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T05:41:25.107 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T05:41:25.107 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T05:41:25.107 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T05:41:25.107 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T05:41:25.108 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:41:25.108 DEBUG:teuthology.orchestra.run.smithi003:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T05:41:25.124 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 708b9316-0cfd-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T05:41:25.124 DEBUG:teuthology.orchestra.run.smithi003:mon.a> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T05:41:25.166 DEBUG:teuthology.orchestra.run.smithi003:mgr.x> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T05:41:25.209 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T05:41:25.209 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a -v bootstrap --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.3 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:41:25.407 INFO:teuthology.orchestra.run.smithi003.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:41:25.407 INFO:teuthology.orchestra.run.smithi003.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', '-v', 'bootstrap', '--fsid', '708b9316-0cfd-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.3', '--skip-admin-label'] 2024-05-08T05:41:25.407 INFO:teuthology.orchestra.run.smithi003.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T05:41:25.408 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying podman|docker is present... 2024-05-08T05:41:25.444 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:41:25.444 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying lvm2 is present... 2024-05-08T05:41:25.444 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying time synchronization is in place... 2024-05-08T05:41:25.452 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:41:25.452 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:41:25.459 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:41:25.459 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-05-08T05:41:25.466 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-05-08T05:41:25.472 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-05-08T05:41:25.473 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:41:25.473 INFO:teuthology.orchestra.run.smithi003.stdout:Repeating the final host check... 2024-05-08T05:41:25.497 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:41:25.497 INFO:teuthology.orchestra.run.smithi003.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-08T05:41:25.497 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl is present 2024-05-08T05:41:25.497 INFO:teuthology.orchestra.run.smithi003.stdout:lvcreate is present 2024-05-08T05:41:25.504 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:41:25.504 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:41:25.510 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:41:25.510 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-05-08T05:41:25.517 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout enabled 2024-05-08T05:41:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout active 2024-05-08T05:41:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:41:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:Host looks OK 2024-05-08T05:41:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:Cluster fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:Acquiring lock 140264661814720 on /run/cephadm/708b9316-0cfd-11ef-bc97-c7b262605968.lock 2024-05-08T05:41:25.524 INFO:teuthology.orchestra.run.smithi003.stdout:Lock 140264661814720 acquired on /run/cephadm/708b9316-0cfd-11ef-bc97-c7b262605968.lock 2024-05-08T05:41:25.525 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 3300 ... 2024-05-08T05:41:25.525 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying IP 172.21.15.3 port 6789 ... 2024-05-08T05:41:25.525 INFO:teuthology.orchestra.run.smithi003.stdout:Base mon IP(s) is [172.21.15.3:3300, 172.21.15.3:6789], mon addrv is [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-05-08T05:41:25.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.3 metric 100 2024-05-08T05:41:25.528 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.3 metric 100 2024-05-08T05:41:25.531 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T05:41:25.531 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-08T05:41:25.531 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-08T05:41:25.534 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T05:41:25.534 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T05:41:25.534 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:41:25.534 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T05:41:25.534 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-05-08T05:41:25.534 INFO:teuthology.orchestra.run.smithi003.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:41:25.535 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-05-08T05:41:25.535 INFO:teuthology.orchestra.run.smithi003.stdout:Mon IP `172.21.15.3` is in CIDR network `172.21.0.0/20` 2024-05-08T05:41:25.535 INFO:teuthology.orchestra.run.smithi003.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T05:41:25.535 INFO:teuthology.orchestra.run.smithi003.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T05:41:25.536 INFO:teuthology.orchestra.run.smithi003.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stdout 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:f08a3efca46759c0efc03dda6bb4c9adf7daf77004822005f2f0da3e8d1877bb 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Copying config sha256:352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:41:25.976 INFO:teuthology.orchestra.run.smithi003.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T05:41:26.765 INFO:teuthology.orchestra.run.smithi003.stdout:ceph: stdout ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:41:26.765 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph version: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:41:26.765 INFO:teuthology.orchestra.run.smithi003.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T05:41:27.756 INFO:teuthology.orchestra.run.smithi003.stdout:stat: stdout 167 167 2024-05-08T05:41:27.756 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial keys... 2024-05-08T05:41:28.677 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQCIEDtmH2KfBhAA9qD0tAK6py0bdlcpvBzWkg== 2024-05-08T05:41:29.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQCIEDtmYsZuOhAAL9DlldO3JXyHWta7BfXwLA== 2024-05-08T05:41:30.522 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph-authtool: stdout AQCKEDtmz5sjABAA3eF8SF9WmOWu59aa/lnGaQ== 2024-05-08T05:41:30.522 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial monmap... 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:monmaptool for a [v2:172.21.15.3:3300,v1:172.21.15.3:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:setting min_mon_release = quincy 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: set fsid to 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:41:31.478 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:41:31.479 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mon... 2024-05-08T05:41:32.425 INFO:teuthology.orchestra.run.smithi003.stdout:create mon.a on 2024-05-08T05:41:32.643 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-08T05:41:32.798 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T05:41:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-708b9316-0cfd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-708b9316-0cfd-11ef-bc97-c7b262605968.target. 2024-05-08T05:41:32.995 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-708b9316-0cfd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-708b9316-0cfd-11ef-bc97-c7b262605968.target. 2024-05-08T05:41:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a 2024-05-08T05:41:33.242 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T05:41:33.410 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-708b9316-0cfd-11ef-bc97-c7b262605968.target.wants/ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-708b9316-0cfd-11ef-bc97-c7b262605968@.service. 2024-05-08T05:41:33.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:33 smithi003 podman[29747]: 2024-05-08 05:41:33.665716137 +0000 UTC m=+0.120445609 container create 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-08T05:41:34.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:33 smithi003 podman[29747]: 2024-05-08 05:41:33.87622778 +0000 UTC m=+0.330957259 container init 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:34.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:33 smithi003 podman[29747]: 2024-05-08 05:41:33.879701334 +0000 UTC m=+0.334430810 container start 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:34.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:33 smithi003 ceph-mon[29766]: mkfs 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:34.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:33 smithi003 ceph-mon[29766]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:41:34.081 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:41:34.082 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-05-08T05:41:34.089 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:41:34.089 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-05-08T05:41:34.089 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-05-08T05:41:34.089 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:41:34.089 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon to start... 2024-05-08T05:41:34.090 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mon... 2024-05-08T05:41:34.313 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 bash[29747]: 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 2024-05-08T05:41:34.313 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 systemd[1]: Started Ceph mon.a for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: monmap epoch 1 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: last_changed 2024-05-08T05:41:30.871663+0000 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: min_mon_release 19 (squid) 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: election_strategy: 1 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: fsmap 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:41:35.108 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: mgrmap e1: no daemons active 2024-05-08T05:41:35.109 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:34 smithi003 ceph-mon[29766]: from='client.? 172.21.15.3:0/2020885597' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T05:41:35.109 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T05:41:35.115 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout id: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout services: 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.675064s) 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout data: 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T05:41:35.116 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.514+0000 7f4420e66700 1 Processor -- start 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.514+0000 7f4420e66700 1 -- start start 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.514+0000 7f4420e66700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.514+0000 7f4420e66700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f441c07d6a0 con 0x7f441c107690 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36662/0 (socket says 172.21.15.3:36662) 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441a59c700 1 -- 172.21.15.3:0/3315662620 learned_addr learned my addr 172.21.15.3:0/3315662620 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441a59c700 1 -- 172.21.15.3:0/3315662620 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c107ff0 con 0x7f441c107690 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441a59c700 1 --2- 172.21.15.3:0/3315662620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c107ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f440c0098a0 tx=0x7f440c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5195f599555c8f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441959a700 1 -- 172.21.15.3:0/3315662620 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f440c004160 con 0x7f441c107690 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441959a700 1 -- 172.21.15.3:0/3315662620 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c004300 con 0x7f441c107690 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.515+0000 7f441959a700 1 -- 172.21.15.3:0/3315662620 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f440c0036a0 con 0x7f441c107690 2024-05-08T05:41:35.117 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 -- 172.21.15.3:0/3315662620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 msgr2=0x7f441c107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 --2- 172.21.15.3:0/3315662620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c107ab0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f440c0098a0 tx=0x7f440c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 -- 172.21.15.3:0/3315662620 shutdown_connections 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 --2- 172.21.15.3:0/3315662620 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c107ab0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 -- 172.21.15.3:0/3315662620 >> 172.21.15.3:0/3315662620 conn(0x7f441c103250 msgr2=0x7f441c105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 -- 172.21.15.3:0/3315662620 shutdown_connections 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.516+0000 7f4420e66700 1 -- 172.21.15.3:0/3315662620 wait complete. 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f4420e66700 1 Processor -- start 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f4420e66700 1 -- start start 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f4420e66700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f4420e66700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f441c108520 con 0x7f441c107690 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f441a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f441a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36674/0 (socket says 172.21.15.3:36674) 2024-05-08T05:41:35.118 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f441a59c700 1 -- 172.21.15.3:0/2020885597 learned_addr learned my addr 172.21.15.3:0/2020885597 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.517+0000 7f441a59c700 1 -- 172.21.15.3:0/2020885597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c19ebf0 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f441a59c700 1 --2- 172.21.15.3:0/2020885597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c19e6b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f440c000c00 tx=0x7f440c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f440c003db0 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f440c011630 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c19ee80 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f440c011950 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c1a1b70 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f440c011af0 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.518+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f440c01c790 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.519+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441c064b00 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.521+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f440c032020 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.594+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f441c1a2060 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.594+0000 7f440b7fe700 1 -- 172.21.15.3:0/2020885597 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f440c004260 con 0x7f441c107690 2024-05-08T05:41:35.119 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 msgr2=0x7f441c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 --2- 172.21.15.3:0/2020885597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c19e6b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f440c000c00 tx=0x7f440c004750 comp rx=0 tx=0).stop 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 shutdown_connections 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 --2- 172.21.15.3:0/2020885597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f441c107690 0x7f441c19e6b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 >> 172.21.15.3:0/2020885597 conn(0x7f441c103250 msgr2=0x7f441c194cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 shutdown_connections 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:34.595+0000 7f4420e66700 1 -- 172.21.15.3:0/2020885597 wait complete. 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:mon is available 2024-05-08T05:41:35.120 INFO:teuthology.orchestra.run.smithi003.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T05:41:36.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:35 smithi003 ceph-mon[29766]: from='client.? 172.21.15.3:0/3908588590' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:41:36.388 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:35 smithi003 ceph-mon[29766]: from='client.? 172.21.15.3:0/3908588590' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.3:3300,v1:172.21.15.3:6789] 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:41:36.398 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f5881463700 1 Processor -- start 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f5881463700 1 -- start start 2024-05-08T05:41:36.399 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f5881463700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f5881463700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f587c1097d0 con 0x7f587c108de0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f587affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f587affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36682/0 (socket says 172.21.15.3:36682) 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.564+0000 7f587affd700 1 -- 172.21.15.3:0/1916703396 learned_addr learned my addr 172.21.15.3:0/1916703396 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f587affd700 1 -- 172.21.15.3:0/1916703396 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c109fe0 con 0x7f587c108de0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f587affd700 1 --2- 172.21.15.3:0/1916703396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c109200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f586c0098a0 tx=0x7f586c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1726b2717e8002b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f5879ffb700 1 -- 172.21.15.3:0/1916703396 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586c004160 con 0x7f587c108de0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f5879ffb700 1 -- 172.21.15.3:0/1916703396 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c004300 con 0x7f587c108de0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f5879ffb700 1 -- 172.21.15.3:0/1916703396 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586c0036a0 con 0x7f587c108de0 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f5881463700 1 -- 172.21.15.3:0/1916703396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 msgr2=0x7f587c109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f5881463700 1 --2- 172.21.15.3:0/1916703396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c109200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f586c0098a0 tx=0x7f586c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.565+0000 7f5881463700 1 -- 172.21.15.3:0/1916703396 shutdown_connections 2024-05-08T05:41:36.400 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.566+0000 7f5881463700 1 --2- 172.21.15.3:0/1916703396 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c109200 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.566+0000 7f5881463700 1 -- 172.21.15.3:0/1916703396 >> 172.21.15.3:0/1916703396 conn(0x7f587c1043a0 msgr2=0x7f587c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.566+0000 7f5881463700 1 -- 172.21.15.3:0/1916703396 shutdown_connections 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.566+0000 7f5881463700 1 -- 172.21.15.3:0/1916703396 wait complete. 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.566+0000 7f5881463700 1 Processor -- start 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.566+0000 7f5881463700 1 -- start start 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5881463700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5881463700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f587c10a510 con 0x7f587c108de0 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f587affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f587affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36686/0 (socket says 172.21.15.3:36686) 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f587affd700 1 -- 172.21.15.3:0/3908588590 learned_addr learned my addr 172.21.15.3:0/3908588590 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f587affd700 1 -- 172.21.15.3:0/3908588590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c19ec10 con 0x7f587c108de0 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f587affd700 1 --2- 172.21.15.3:0/3908588590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c19e6d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f586c000c00 tx=0x7f586c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586c003db0 con 0x7f587c108de0 2024-05-08T05:41:36.401 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f586c011630 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c19eea0 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586c011950 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c1a1b90 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.567+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f586c011af0 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.568+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f586c01c790 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.568+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c064b00 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.571+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f586c032020 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.642+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f587c1a1d60 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.645+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+463 (secure 0 0 0) 0x7f586c004260 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.646+0000 7f5863fff700 1 -- 172.21.15.3:0/3908588590 <== mon.0 v2:172.21.15.3:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f586c01ce20 con 0x7f587c108de0 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.646+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 msgr2=0x7f587c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.646+0000 7f5881463700 1 --2- 172.21.15.3:0/3908588590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c19e6d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f586c000c00 tx=0x7f586c004750 comp rx=0 tx=0).stop 2024-05-08T05:41:36.402 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.646+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 shutdown_connections 2024-05-08T05:41:36.403 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.646+0000 7f5881463700 1 --2- 172.21.15.3:0/3908588590 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f587c108de0 0x7f587c19e6d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:36.403 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.646+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 >> 172.21.15.3:0/3908588590 conn(0x7f587c1043a0 msgr2=0x7f587c07eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:36.403 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.647+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 shutdown_connections 2024-05-08T05:41:36.403 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:35.647+0000 7f5881463700 1 -- 172.21.15.3:0/3908588590 wait complete. 2024-05-08T05:41:36.403 INFO:teuthology.orchestra.run.smithi003.stdout:Generating new minimal ceph.conf... 2024-05-08T05:41:37.219 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:36 smithi003 ceph-mon[29766]: from='client.? 172.21.15.3:0/1532712759' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effe58de700 1 Processor -- start 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effe58de700 1 -- start start 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effe58de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe007dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effe58de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effe007e180 con 0x7effe007f750 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effdeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe007dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effdeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe007dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36702/0 (socket says 172.21.15.3:36702) 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.834+0000 7effdeffd700 1 -- 172.21.15.3:0/1468486241 learned_addr learned my addr 172.21.15.3:0/1468486241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:37.485 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.835+0000 7effdeffd700 1 -- 172.21.15.3:0/1468486241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe007e300 con 0x7effe007f750 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.835+0000 7effdeffd700 1 --2- 172.21.15.3:0/1468486241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe007dbb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7effd00098a0 tx=0x7effd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc802803d1890fd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.835+0000 7effddffb700 1 -- 172.21.15.3:0/1468486241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0004160 con 0x7effe007f750 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.835+0000 7effddffb700 1 -- 172.21.15.3:0/1468486241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7effd0004300 con 0x7effe007f750 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.835+0000 7effddffb700 1 -- 172.21.15.3:0/1468486241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0011af0 con 0x7effe007f750 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.835+0000 7effe58de700 1 -- 172.21.15.3:0/1468486241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 msgr2=0x7effe007dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 --2- 172.21.15.3:0/1468486241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe007dbb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7effd00098a0 tx=0x7effd0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 -- 172.21.15.3:0/1468486241 shutdown_connections 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 --2- 172.21.15.3:0/1468486241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe007dbb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 -- 172.21.15.3:0/1468486241 >> 172.21.15.3:0/1468486241 conn(0x7effe01043a0 msgr2=0x7effe01067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 -- 172.21.15.3:0/1468486241 shutdown_connections 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 -- 172.21.15.3:0/1468486241 wait complete. 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 Processor -- start 2024-05-08T05:41:37.486 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.836+0000 7effe58de700 1 -- start start 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe58de700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe58de700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7effe010f3a0 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effdeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effdeffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36706/0 (socket says 172.21.15.3:36706) 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effdeffd700 1 -- 172.21.15.3:0/1532712759 learned_addr learned my addr 172.21.15.3:0/1532712759 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effdeffd700 1 -- 172.21.15.3:0/1532712759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effe019f490 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effdeffd700 1 --2- 172.21.15.3:0/1532712759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe019ef50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c00 tx=0x7effd0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0003b20 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7effd001a8c0 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effe019f720 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7effd0024940 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.837+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effe01a2410 con 0x7effe007f750 2024-05-08T05:41:37.487 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.838+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7effd0022020 con 0x7effe007f750 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.838+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7effd002ddd0 con 0x7effe007f750 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.838+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effe0064b00 con 0x7effe007f750 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.841+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7effd0042ec0 con 0x7effe007f750 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.912+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7effe01a2950 con 0x7effe007f750 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.913+0000 7effe48dc700 1 -- 172.21.15.3:0/1532712759 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7effd0031020 con 0x7effe007f750 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.913+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 msgr2=0x7effe019ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.913+0000 7effe58de700 1 --2- 172.21.15.3:0/1532712759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe019ef50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7effd0000c00 tx=0x7effd0011b70 comp rx=0 tx=0).stop 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.913+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 shutdown_connections 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.913+0000 7effe58de700 1 --2- 172.21.15.3:0/1532712759 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7effe007f750 0x7effe019ef50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.913+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 >> 172.21.15.3:0/1532712759 conn(0x7effe01043a0 msgr2=0x7effe0104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.914+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 shutdown_connections 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:36.914+0000 7effe58de700 1 -- 172.21.15.3:0/1532712759 wait complete. 2024-05-08T05:41:37.488 INFO:teuthology.orchestra.run.smithi003.stdout:Restarting the monitor... 2024-05-08T05:41:37.765 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:37 smithi003 systemd[1]: Stopping Ceph mon.a for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:41:38.114 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:37 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a[29762]: 2024-05-08T05:41:37.764+0000 7fe2f6acf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:38.115 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:37 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a[29762]: 2024-05-08T05:41:37.764+0000 7fe2f6acf700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T05:41:38.115 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:37 smithi003 podman[30012]: 2024-05-08 05:41:37.867518437 +0000 UTC m=+0.210988927 container died 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:38.372 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 podman[30012]: 2024-05-08 05:41:38.197400463 +0000 UTC m=+0.540870941 container cleanup 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-08T05:41:38.373 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 bash[30012]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a 2024-05-08T05:41:38.622 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 podman[30026]: 2024-05-08 05:41:38.373216061 +0000 UTC m=+0.502621501 container remove 24dd23fd9ca8f9a226f83f9b2bbf9e768456bc5e8ae265b2bdc246c468799ad8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T05:41:38.623 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-08T05:41:38.623 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 systemd[1]: Stopped Ceph mon.a for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:41:38.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 systemd[1]: Starting Ceph mon.a for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:41:38.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 podman[30117]: 2024-05-08 05:41:38.744344712 +0000 UTC m=+0.018539945 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:41:38.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:38 smithi003 podman[30117]: 2024-05-08 05:41:38.852265105 +0000 UTC m=+0.126460336 container create 25a2aff035ee0634c0905d1a3098821605dac2b066ea8bcd6b0f9e26cc20dc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-08T05:41:39.138 INFO:teuthology.orchestra.run.smithi003.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T05:41:39.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 podman[30117]: 2024-05-08 05:41:39.059773939 +0000 UTC m=+0.333969171 container init 25a2aff035ee0634c0905d1a3098821605dac2b066ea8bcd6b0f9e26cc20dc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:39.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 podman[30117]: 2024-05-08 05:41:39.062996358 +0000 UTC m=+0.337191590 container start 25a2aff035ee0634c0905d1a3098821605dac2b066ea8bcd6b0f9e26cc20dc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T05:41:39.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:41:39.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:41:39.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: pidfile_write: ignore empty --pid-file 2024-05-08T05:41:39.236 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: load: jerasure load: lrc 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Git sha 0 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: DB SUMMARY 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: DB Session ID: CJV9WMDNMCZWRQZLXR31 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104484 ; 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:41:39.237 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.env: 0x555573f29cc0 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.info_log: 0x555575aefda0 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:41:39.238 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.statistics: (nil) 2024-05-08T05:41:39.239 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.use_fsync: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.db_log_dir: 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.wal_dir: 2024-05-08T05:41:39.240 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.write_buffer_manager: 0x555575ac10e0 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:41:39.241 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.unordered_write: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.row_cache: None 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.wal_filter: None 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:41:39.242 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.wal_compression: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:41:39.243 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_open_files: -1 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:41:39.244 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Compression algorithms supported: 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kZSTD supported: 0 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:41:39.245 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.merge_operator: 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_filter: None 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555575aeff60) 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_type: 0 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_index_type: 0 2024-05-08T05:41:39.246 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_shortening: 1 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: checksum: 4 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: no_block_cache: 0 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache: 0x555575af31f0 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_options: 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: capacity : 536870912 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_shard_bits : 4 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: strict_capacity_limit : 0 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_cache_compressed: (nil) 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: persistent_cache: (nil) 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size: 4096 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_size_deviation: 10 2024-05-08T05:41:39.247 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_restart_interval: 16 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: index_block_restart_interval: 1 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: metadata_block_size: 4096 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: partition_filters: 0 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: use_delta_encoding: 1 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: filter_policy: bloomfilter 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: whole_key_filtering: 1 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: verify_compression: 0 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: format_version: 5 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_index_compression: 1 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: block_align: 0 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: max_auto_readahead_size: 262144 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: prepopulate_block_cache: 0 2024-05-08T05:41:39.248 INFO:journalctl@ceph.mon.a.smithi003.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression: NoCompression 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.num_levels: 7 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:41:39.249 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:41:39.250 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:41:39.251 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:41:39.252 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:41:39.253 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.ttl: 2592000 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:41:39.254 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8015afb2-08dc-4be2-8f4d-f64558484996 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146899095154, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146899096259, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99711, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97593, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14028, "raw_average_key_size": 49, "raw_value_size": 89951, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715146899, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8015afb2-08dc-4be2-8f4d-f64558484996", "db_session_id": "CJV9WMDNMCZWRQZLXR31", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146899096347, "job": 1, "event": "recovery_finished"} 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555575c10000 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: DB pointer 0x555575bfc000 2024-05-08T05:41:39.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** DB Stats ** 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: L0 2/0 99.23 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:41:39.256 INFO:journalctl@ceph.mon.a.smithi003.stdout: Sum 2/0 99.23 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 137.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** Compaction Stats [default] ** 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 137.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:41:39.257 INFO:journalctl@ceph.mon.a.smithi003.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: Cumulative compaction: 0.00 GB write, 27.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: Interval compaction: 0.00 GB write, 27.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache BinnedLRUCache@0x555575af31f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: starting mon.a rank 0 at public addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] at bind addrs [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???) e1 preinit fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).mds e1 new map 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).mds e1 print_map 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: e1 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:41:39.258 INFO:journalctl@ceph.mon.a.smithi003.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout: legacy client fscid: -1 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout: 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout: No filesystems configured 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 bash[30117]: 25a2aff035ee0634c0905d1a3098821605dac2b066ea8bcd6b0f9e26cc20dc3c 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 systemd[1]: Started Ceph mon.a for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: monmap epoch 1 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: last_changed 2024-05-08T05:41:30.871663+0000 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:41:39.259 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: min_mon_release 19 (squid) 2024-05-08T05:41:39.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: election_strategy: 1 2024-05-08T05:41:39.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:41:39.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: fsmap 2024-05-08T05:41:39.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:41:39.260 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:39 smithi003 ceph-mon[30137]: mgrmap e1: no daemons active 2024-05-08T05:41:40.176 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.577+0000 7fbe4aa7f700 1 Processor -- start 2024-05-08T05:41:40.181 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.577+0000 7fbe4aa7f700 1 -- start start 2024-05-08T05:41:40.181 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.577+0000 7fbe4aa7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe441080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:40.181 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.577+0000 7fbe4aa7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441086c0 con 0x7fbe44107cd0 2024-05-08T05:41:40.181 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe4881b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe441080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:40.181 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe4881b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe441080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36714/0 (socket says 172.21.15.3:36714) 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe4881b700 1 -- 172.21.15.3:0/2725450665 learned_addr learned my addr 172.21.15.3:0/2725450665 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe4881b700 1 -- 172.21.15.3:0/2725450665 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe44108ed0 con 0x7fbe44107cd0 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe4881b700 1 --2- 172.21.15.3:0/2725450665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe441080f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098a0 tx=0x7fbe38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da001809105e2be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe437fe700 1 -- 172.21.15.3:0/2725450665 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe38004160 con 0x7fbe44107cd0 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe437fe700 1 -- 172.21.15.3:0/2725450665 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe38004300 con 0x7fbe44107cd0 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.578+0000 7fbe437fe700 1 -- 172.21.15.3:0/2725450665 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe38011af0 con 0x7fbe44107cd0 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/2725450665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 msgr2=0x7fbe441080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 --2- 172.21.15.3:0/2725450665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe441080f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098a0 tx=0x7fbe38009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/2725450665 shutdown_connections 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 --2- 172.21.15.3:0/2725450665 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe441080f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/2725450665 >> 172.21.15.3:0/2725450665 conn(0x7fbe441032b0 msgr2=0x7fbe441056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/2725450665 shutdown_connections 2024-05-08T05:41:40.182 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.579+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/2725450665 wait complete. 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4aa7f700 1 Processor -- start 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4aa7f700 1 -- start start 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4aa7f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe44192e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4aa7f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410c280 con 0x7fbe44107cd0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4881b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe44192e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4881b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe44192e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36730/0 (socket says 172.21.15.3:36730) 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4881b700 1 -- 172.21.15.3:0/1689487029 learned_addr learned my addr 172.21.15.3:0/1689487029 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4881b700 1 -- 172.21.15.3:0/1689487029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe44193380 con 0x7fbe44107cd0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.580+0000 7fbe4881b700 1 --2- 172.21.15.3:0/1689487029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe44192e40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbe38003f40 tx=0x7fbe38011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe380038b0 con 0x7fbe44107cd0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe38003a50 con 0x7fbe44107cd0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe44191570 con 0x7fbe44107cd0 2024-05-08T05:41:40.183 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe3801a5c0 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe44191ab0 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbe3801ac00 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.581+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbe38019700 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.582+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe44064b00 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.584+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbe38019900 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.656+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbe44196750 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.657+0000 7fbe41ffb700 1 -- 172.21.15.3:0/1689487029 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbe38019ae0 con 0x7fbe44107cd0 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 msgr2=0x7fbe44192e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 --2- 172.21.15.3:0/1689487029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe44192e40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbe38003f40 tx=0x7fbe38011890 comp rx=0 tx=0).stop 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 shutdown_connections 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 --2- 172.21.15.3:0/1689487029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbe44107cd0 0x7fbe44192e40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 >> 172.21.15.3:0/1689487029 conn(0x7fbe441032b0 msgr2=0x7fbe44104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 shutdown_connections 2024-05-08T05:41:40.184 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:39.658+0000 7fbe4aa7f700 1 -- 172.21.15.3:0/1689487029 wait complete. 2024-05-08T05:41:40.185 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T05:41:40.198 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:41:40.198 INFO:teuthology.orchestra.run.smithi003.stdout:Creating mgr... 2024-05-08T05:41:40.198 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T05:41:40.199 INFO:teuthology.orchestra.run.smithi003.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T05:41:40.431 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x 2024-05-08T05:41:40.432 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Failed to reset failed state of unit ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T05:41:40.585 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-708b9316-0cfd-11ef-bc97-c7b262605968.target.wants/ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-708b9316-0cfd-11ef-bc97-c7b262605968@.service. 2024-05-08T05:41:40.818 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:40 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1689487029' entity='client.admin' 2024-05-08T05:41:41.078 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 podman[30355]: 2024-05-08 05:41:41.00631564 +0000 UTC m=+0.292445728 container start a09ad788764b7ffe0dc90992ea608c3371dacc7274f212131c9bb68f5996a44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:41.079 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 bash[30355]: a09ad788764b7ffe0dc90992ea608c3371dacc7274f212131c9bb68f5996a44f 2024-05-08T05:41:41.092 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:41:41.092 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-05-08T05:41:41.100 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:41:41.100 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-05-08T05:41:41.100 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-05-08T05:41:41.100 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:41:41.109 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:41:41.109 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-05-08T05:41:41.117 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:41:41.117 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-05-08T05:41:41.117 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-05-08T05:41:41.117 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T05:41:41.117 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr to start... 2024-05-08T05:41:41.117 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr... 2024-05-08T05:41:41.338 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 systemd[1]: Started Ceph mgr.x for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:41:41.338 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:41.174+0000 7f5849fd2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:41:41.628 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:41.372+0000 7f5849fd2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:41.628 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:41.479+0000 7f5849fd2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:41:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:41 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/583493020' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:41:41.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:41.825+0000 7f5849fd2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:41:42.276 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:41.916+0000 7f5849fd2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:41:42.289 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:41:42.290 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:41:42.291 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:41:42.292 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:33.922077+0000", 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.527+0000 7f24cc56d700 1 Processor -- start 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.527+0000 7f24cc56d700 1 -- start start 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.527+0000 7f24cc56d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c410bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.527+0000 7f24cc56d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24c407d720 con 0x7f24c4109800 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.528+0000 7f24ca309700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c410bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.528+0000 7f24ca309700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c410bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36764/0 (socket says 172.21.15.3:36764) 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.528+0000 7f24ca309700 1 -- 172.21.15.3:0/1757643853 learned_addr learned my addr 172.21.15.3:0/1757643853 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.528+0000 7f24ca309700 1 -- 172.21.15.3:0/1757643853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c410c130 con 0x7f24c4109800 2024-05-08T05:41:42.293 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.528+0000 7f24ca309700 1 --2- 172.21.15.3:0/1757643853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c410bbf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f24c00098a0 tx=0x7f24c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02776267ce1510b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.528+0000 7f24c9307700 1 -- 172.21.15.3:0/1757643853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c0004160 con 0x7f24c4109800 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24c9307700 1 -- 172.21.15.3:0/1757643853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24c0004300 con 0x7f24c4109800 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24c9307700 1 -- 172.21.15.3:0/1757643853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c0011af0 con 0x7f24c4109800 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24cc56d700 1 -- 172.21.15.3:0/1757643853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 msgr2=0x7f24c410bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24cc56d700 1 --2- 172.21.15.3:0/1757643853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c410bbf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f24c00098a0 tx=0x7f24c0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24cc56d700 1 -- 172.21.15.3:0/1757643853 shutdown_connections 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24cc56d700 1 --2- 172.21.15.3:0/1757643853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c410bbf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.529+0000 7f24cc56d700 1 -- 172.21.15.3:0/1757643853 >> 172.21.15.3:0/1757643853 conn(0x7f24c4103210 msgr2=0x7f24c4105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.530+0000 7f24cc56d700 1 -- 172.21.15.3:0/1757643853 shutdown_connections 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.530+0000 7f24cc56d700 1 -- 172.21.15.3:0/1757643853 wait complete. 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.530+0000 7f24cc56d700 1 Processor -- start 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.530+0000 7f24cc56d700 1 -- start start 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.530+0000 7f24cc56d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c4109080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:42.294 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.530+0000 7f24cc56d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f24c410d100 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24ca309700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c4109080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24ca309700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c4109080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36774/0 (socket says 172.21.15.3:36774) 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24ca309700 1 -- 172.21.15.3:0/583493020 learned_addr learned my addr 172.21.15.3:0/583493020 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24ca309700 1 -- 172.21.15.3:0/583493020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c41095c0 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24ca309700 1 --2- 172.21.15.3:0/583493020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c4109080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24c0003f40 tx=0x7f24c0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c00038b0 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f24c0003a50 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41077b0 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24c001a5c0 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.531+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c4107cf0 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.532+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f24c001ac50 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.532+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f24c0019700 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.533+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c4064b00 con 0x7f24c4109800 2024-05-08T05:41:42.295 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.537+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f24c0019900 con 0x7f24c4109800 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.625+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f24c4107fa0 con 0x7f24c4109800 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.625+0000 7f24bb7fe700 1 -- 172.21.15.3:0/583493020 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f24c0019ae0 con 0x7f24c4109800 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 msgr2=0x7f24c4109080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 --2- 172.21.15.3:0/583493020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c4109080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f24c0003f40 tx=0x7f24c0011890 comp rx=0 tx=0).stop 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 shutdown_connections 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 --2- 172.21.15.3:0/583493020 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f24c4109800 0x7f24c4109080 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 >> 172.21.15.3:0/583493020 conn(0x7f24c4103210 msgr2=0x7f24c4103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 shutdown_connections 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:41.626+0000 7f24cc56d700 1 -- 172.21.15.3:0/583493020 wait complete. 2024-05-08T05:41:42.296 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (1/15)... 2024-05-08T05:41:43.519 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:43.182+0000 7f5849fd2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:41:43.519 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:43.359+0000 7f5849fd2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:41:43.839 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:43.540+0000 7f5849fd2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:41:43.840 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:43.839+0000 7f5849fd2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:41:44.148 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:43.935+0000 7f5849fd2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:41:44.149 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:44.148+0000 7f5849fd2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:41:44.409 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:44.355+0000 7f5849fd2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:45.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:44.991+0000 7f5849fd2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:41:45.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:45.223+0000 7f5849fd2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:41:45.837 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:45.668+0000 7f5849fd2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:41:45.837 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:45.769+0000 7f5849fd2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:41:46.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:46.154+0000 7f5849fd2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:41:46.689 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:46.543+0000 7f5849fd2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:41:47.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:46.643+0000 7f5849fd2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:41:47.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:46.904+0000 7f5849fd2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:41:47.048 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:47.046+0000 7f5849fd2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:41:47.336 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:47.146+0000 7f5849fd2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:41:47.592 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:47 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3917825008' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:41:47.592 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:47.494+0000 7f5849fd2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:41:47.592 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:47.590+0000 7f5849fd2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:41:48.451 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:48 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:48.224+0000 7f5849fd2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:41:48.452 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:48 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:48.451+0000 7f5849fd2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:41:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: Activating manager daemon x 2024-05-08T05:41:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: mgrmap e2: x(active, starting, since 0.00318634s) 2024-05-08T05:41:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:41:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:41:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:41:48.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:41:48.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:41:48.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:48 smithi003 ceph-mon[30137]: Manager daemon x is now available 2024-05-08T05:41:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:49 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:41:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:49 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:41:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:49 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' 2024-05-08T05:41:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:49 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' 2024-05-08T05:41:49.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:49 smithi003 ceph-mon[30137]: from='mgr.14100 172.21.15.3:0/893921184' entity='mgr.x' 2024-05-08T05:41:49.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:49 smithi003 ceph-mon[30137]: mgrmap e3: x(active, since 1.00797s) 2024-05-08T05:41:51.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:51 smithi003 ceph-mon[30137]: mgrmap e4: x(active, since 2s) 2024-05-08T05:41:53.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:53.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:41:53.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:41:53.406 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:41:53.407 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:41:53.408 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:41:53.409 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:33.922077+0000", 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:53.410 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.334+0000 7f187d485700 1 Processor -- start 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.334+0000 7f187d485700 1 -- start start 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.334+0000 7f187d485700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f187810bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.334+0000 7f187d485700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f187807d720 con 0x7f1878109810 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f187810bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f187810bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41454/0 (socket says 172.21.15.3:41454) 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1876ffd700 1 -- 172.21.15.3:0/2547822835 learned_addr learned my addr 172.21.15.3:0/2547822835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1876ffd700 1 -- 172.21.15.3:0/2547822835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187810c140 con 0x7f1878109810 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1876ffd700 1 --2- 172.21.15.3:0/2547822835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f187810bc00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18600098a0 tx=0x7f1860009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17a4ac1c19e59da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1875ffb700 1 -- 172.21.15.3:0/2547822835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1860004160 con 0x7f1878109810 2024-05-08T05:41:53.411 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1875ffb700 1 -- 172.21.15.3:0/2547822835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1860004300 con 0x7f1878109810 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.335+0000 7f1875ffb700 1 -- 172.21.15.3:0/2547822835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1860011af0 con 0x7f1878109810 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 -- 172.21.15.3:0/2547822835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 msgr2=0x7f187810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 --2- 172.21.15.3:0/2547822835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f187810bc00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18600098a0 tx=0x7f1860009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 -- 172.21.15.3:0/2547822835 shutdown_connections 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 --2- 172.21.15.3:0/2547822835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f187810bc00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 -- 172.21.15.3:0/2547822835 >> 172.21.15.3:0/2547822835 conn(0x7f1878103280 msgr2=0x7f18781056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 -- 172.21.15.3:0/2547822835 shutdown_connections 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.336+0000 7f187d485700 1 -- 172.21.15.3:0/2547822835 wait complete. 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.337+0000 7f187d485700 1 Processor -- start 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.337+0000 7f187d485700 1 -- start start 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.337+0000 7f187d485700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f1878109090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.337+0000 7f187d485700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f187810d110 con 0x7f1878109810 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.337+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f1878109090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:53.412 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f1876ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f1878109090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41460/0 (socket says 172.21.15.3:41460) 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f1876ffd700 1 -- 172.21.15.3:0/3917825008 learned_addr learned my addr 172.21.15.3:0/3917825008 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f1876ffd700 1 -- 172.21.15.3:0/3917825008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18781095d0 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f1876ffd700 1 --2- 172.21.15.3:0/3917825008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f1878109090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1860011ac0 tx=0x7f1860011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1860003b20 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18781077c0 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f186001a8c0 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18600248d0 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1878107d00 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1860022020 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.338+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18600193c0 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.339+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1878064b00 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.342+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f186001aa70 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.418+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1878107fb0 con 0x7f1878109810 2024-05-08T05:41:53.413 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.419+0000 7f186ffff700 1 -- 172.21.15.3:0/3917825008 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f186001a430 con 0x7f1878109810 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.419+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 msgr2=0x7f1878109090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.419+0000 7f187d485700 1 --2- 172.21.15.3:0/3917825008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f1878109090 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1860011ac0 tx=0x7f1860011ba0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.420+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 shutdown_connections 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.420+0000 7f187d485700 1 --2- 172.21.15.3:0/3917825008 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1878109810 0x7f1878109090 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.420+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 >> 172.21.15.3:0/3917825008 conn(0x7f1878103280 msgr2=0x7f1878103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.420+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 shutdown_connections 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:47.420+0000 7f187d485700 1 -- 172.21.15.3:0/3917825008 wait complete. 2024-05-08T05:41:53.414 INFO:teuthology.orchestra.run.smithi003.stdout:mgr not available, waiting (2/15)... 2024-05-08T05:41:56.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:56 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2588029746' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:41:56.678 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:56.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:41:56.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:41:56.681 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:41:56.682 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:41:56.683 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:41:56.693 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:41:56.694 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:41:56.695 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:41:33.922077+0000", 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622c4a4700 1 Processor -- start 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622c4a4700 1 -- start start 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622c4a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f622410bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622c4a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f622407d700 con 0x7f6224109840 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622a240700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f622410bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622a240700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f622410bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57756/0 (socket says 172.21.15.3:57756) 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.821+0000 7f622a240700 1 -- 172.21.15.3:0/2710370866 learned_addr learned my addr 172.21.15.3:0/2710370866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:56.696 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.822+0000 7f622a240700 1 -- 172.21.15.3:0/2710370866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622410c170 con 0x7f6224109840 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.822+0000 7f622a240700 1 --2- 172.21.15.3:0/2710370866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f622410bc30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f62200098a0 tx=0x7f6220009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71caf65640bb3579 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.822+0000 7f622923e700 1 -- 172.21.15.3:0/2710370866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6220004160 con 0x7f6224109840 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.822+0000 7f622923e700 1 -- 172.21.15.3:0/2710370866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6220004300 con 0x7f6224109840 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.822+0000 7f622923e700 1 -- 172.21.15.3:0/2710370866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6220011af0 con 0x7f6224109840 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 -- 172.21.15.3:0/2710370866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 msgr2=0x7f622410bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 --2- 172.21.15.3:0/2710370866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f622410bc30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f62200098a0 tx=0x7f6220009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 -- 172.21.15.3:0/2710370866 shutdown_connections 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 --2- 172.21.15.3:0/2710370866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f622410bc30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 -- 172.21.15.3:0/2710370866 >> 172.21.15.3:0/2710370866 conn(0x7f6224103270 msgr2=0x7f6224105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 -- 172.21.15.3:0/2710370866 shutdown_connections 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.823+0000 7f622c4a4700 1 -- 172.21.15.3:0/2710370866 wait complete. 2024-05-08T05:41:56.697 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622c4a4700 1 Processor -- start 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622c4a4700 1 -- start start 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622c4a4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f62241090c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622c4a4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f622410d0c0 con 0x7f6224109840 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622a240700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f62241090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622a240700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f62241090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57766/0 (socket says 172.21.15.3:57766) 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622a240700 1 -- 172.21.15.3:0/2588029746 learned_addr learned my addr 172.21.15.3:0/2588029746 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622a240700 1 -- 172.21.15.3:0/2588029746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6224109600 con 0x7f6224109840 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.824+0000 7f622a240700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f62241090c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6220003f40 tx=0x7f6220011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.825+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62200038b0 con 0x7f6224109840 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.825+0000 7f622c4a4700 1 -- 172.21.15.3:0/2588029746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62241077f0 con 0x7f6224109840 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.825+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6220003a50 con 0x7f6224109840 2024-05-08T05:41:56.698 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.825+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f622001a5c0 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.825+0000 7f622c4a4700 1 -- 172.21.15.3:0/2588029746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6224107d30 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.826+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224064b00 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.827+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f622001a870 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.827+0000 7f62177fe700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f621003e420 0x7f62100408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.827+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6220053c00 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.827+0000 7f6229a3f700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f621003e420 0x7f62100408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.828+0000 7f6229a3f700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f621003e420 0x7f62100408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6218006fd0 tx=0x7f6218006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:55.832+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f622000b3e0 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.149+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f62241a4550 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.150+0000 7f62177fe700 1 -- 172.21.15.3:0/2588029746 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f62200526c0 con 0x7f6224109840 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.152+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f621003e420 msgr2=0x7f62100408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:56.699 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.152+0000 7f62157fa700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f621003e420 0x7f62100408e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6218006fd0 tx=0x7f6218006e40 comp rx=0 tx=0).stop 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.152+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 msgr2=0x7f62241090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.152+0000 7f62157fa700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f62241090c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6220003f40 tx=0x7f6220011890 comp rx=0 tx=0).stop 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.152+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 shutdown_connections 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.152+0000 7f62157fa700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f621003e420 0x7f62100408e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.153+0000 7f62157fa700 1 --2- 172.21.15.3:0/2588029746 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6224109840 0x7f62241090c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.153+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 >> 172.21.15.3:0/2588029746 conn(0x7f6224103270 msgr2=0x7f6224103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.153+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 shutdown_connections 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:56.153+0000 7f62157fa700 1 -- 172.21.15.3:0/2588029746 wait complete. 2024-05-08T05:41:56.700 INFO:teuthology.orchestra.run.smithi003.stdout:mgr is available 2024-05-08T05:41:57.647 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:57 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1086515012' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout fsid = 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:41:57.900 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.115+0000 7ff4a36d0700 1 Processor -- start 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.115+0000 7ff4a36d0700 1 -- start start 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.115+0000 7ff4a36d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c1091d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.115+0000 7ff4a36d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c1097a0 con 0x7ff49c108db0 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.115+0000 7ff4a146c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c1091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff4a146c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c1091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57768/0 (socket says 172.21.15.3:57768) 2024-05-08T05:41:57.901 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff4a146c700 1 -- 172.21.15.3:0/2998414581 learned_addr learned my addr 172.21.15.3:0/2998414581 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff4a146c700 1 -- 172.21.15.3:0/2998414581 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c109fb0 con 0x7ff49c108db0 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff4a146c700 1 --2- 172.21.15.3:0/2998414581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c1091d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249e179a7bc78b05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff493fff700 1 -- 172.21.15.3:0/2998414581 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498004160 con 0x7ff49c108db0 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff493fff700 1 -- 172.21.15.3:0/2998414581 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff498004300 con 0x7ff49c108db0 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.116+0000 7ff493fff700 1 -- 172.21.15.3:0/2998414581 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff498011af0 con 0x7ff49c108db0 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 -- 172.21.15.3:0/2998414581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 msgr2=0x7ff49c1091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 --2- 172.21.15.3:0/2998414581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c1091d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff4980098a0 tx=0x7ff498009bb0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 -- 172.21.15.3:0/2998414581 shutdown_connections 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 --2- 172.21.15.3:0/2998414581 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c1091d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 -- 172.21.15.3:0/2998414581 >> 172.21.15.3:0/2998414581 conn(0x7ff49c104330 msgr2=0x7ff49c106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 -- 172.21.15.3:0/2998414581 shutdown_connections 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.117+0000 7ff4a36d0700 1 -- 172.21.15.3:0/2998414581 wait complete. 2024-05-08T05:41:57.902 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a36d0700 1 Processor -- start 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a36d0700 1 -- start start 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a36d0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a36d0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10d360 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a146c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a146c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57776/0 (socket says 172.21.15.3:57776) 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a146c700 1 -- 172.21.15.3:0/1086515012 learned_addr learned my addr 172.21.15.3:0/1086515012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.118+0000 7ff4a146c700 1 -- 172.21.15.3:0/1086515012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c19d2a0 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.119+0000 7ff4a146c700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c19cd60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff498003f40 tx=0x7ff498011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.119+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4980038b0 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.119+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff498003a50 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.119+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c19d530 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.119+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff49801a5c0 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.119+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a0220 con 0x7ff49c108db0 2024-05-08T05:41:57.903 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.120+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7ff49801a8c0 con 0x7ff49c108db0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.120+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c064b00 con 0x7ff49c108db0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.120+0000 7ff4927fc700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7ff48803e3d0 0x7ff488040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.120+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff498053ad0 con 0x7ff49c108db0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.120+0000 7ff4a0c6b700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7ff48803e3d0 0x7ff488040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.121+0000 7ff4a0c6b700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7ff48803e3d0 0x7ff488040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff48c006fd0 tx=0x7ff48c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.126+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff49800e260 con 0x7ff49c108db0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.339+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff49c1a03f0 con 0x7ff49c108db0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.340+0000 7ff4927fc700 1 -- 172.21.15.3:0/1086515012 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7ff49c1a03f0 con 0x7ff49c108db0 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.342+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7ff48803e3d0 msgr2=0x7ff488040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.342+0000 7ff4a36d0700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7ff48803e3d0 0x7ff488040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff48c006fd0 tx=0x7ff48c006e40 comp rx=0 tx=0).stop 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.342+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 msgr2=0x7ff49c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.342+0000 7ff4a36d0700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c19cd60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff498003f40 tx=0x7ff498011890 comp rx=0 tx=0).stop 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.343+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 shutdown_connections 2024-05-08T05:41:57.904 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.343+0000 7ff4a36d0700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7ff48803e3d0 0x7ff488040890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.343+0000 7ff4a36d0700 1 --2- 172.21.15.3:0/1086515012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff49c108db0 0x7ff49c19cd60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.343+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 >> 172.21.15.3:0/1086515012 conn(0x7ff49c104330 msgr2=0x7ff49c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.343+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 shutdown_connections 2024-05-08T05:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:57.343+0000 7ff4a36d0700 1 -- 172.21.15.3:0/1086515012 wait complete. 2024-05-08T05:41:57.905 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling cephadm module... 2024-05-08T05:41:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:58 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/8505228' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T05:41:58.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: ignoring --setuser ceph since I am not root 2024-05-08T05:41:58.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: ignoring --setgroup ceph since I am not root 2024-05-08T05:41:58.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:58.828+0000 7f6af9e7a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:41:59.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:59.025+0000 7f6af9e7a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:41:59.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:59.134+0000 7f6af9e7a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:41:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:59 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/8505228' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T05:41:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:41:59 smithi003 ceph-mon[30137]: mgrmap e5: x(active, since 10s) 2024-05-08T05:41:59.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:59.456+0000 7f6af9e7a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:41:59.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:41:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:41:59.546+0000 7f6af9e7a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:42:01.086 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:00.803+0000 7f6af9e7a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:42:01.087 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:00.981+0000 7f6af9e7a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:01.095 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc766bc5700 1 Processor -- start 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc766bc5700 1 -- start start 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc766bc5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc760109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc766bc5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7601097d0 con 0x7fc760108de0 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc764961700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc760109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc764961700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc760109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57788/0 (socket says 172.21.15.3:57788) 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.314+0000 7fc764961700 1 -- 172.21.15.3:0/4070401819 learned_addr learned my addr 172.21.15.3:0/4070401819 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.315+0000 7fc764961700 1 -- 172.21.15.3:0/4070401819 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc760109fe0 con 0x7fc760108de0 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.315+0000 7fc764961700 1 --2- 172.21.15.3:0/4070401819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc760109200 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7540098a0 tx=0x7fc754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaadb90a3c21bb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.096 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.315+0000 7fc75f7fe700 1 -- 172.21.15.3:0/4070401819 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc754004160 con 0x7fc760108de0 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.315+0000 7fc75f7fe700 1 -- 172.21.15.3:0/4070401819 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc754004300 con 0x7fc760108de0 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.315+0000 7fc75f7fe700 1 -- 172.21.15.3:0/4070401819 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc754011af0 con 0x7fc760108de0 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 -- 172.21.15.3:0/4070401819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 msgr2=0x7fc760109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 --2- 172.21.15.3:0/4070401819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc760109200 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc7540098a0 tx=0x7fc754009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 -- 172.21.15.3:0/4070401819 shutdown_connections 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 --2- 172.21.15.3:0/4070401819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc760109200 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 -- 172.21.15.3:0/4070401819 >> 172.21.15.3:0/4070401819 conn(0x7fc7601043a0 msgr2=0x7fc7601067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:01.097 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 -- 172.21.15.3:0/4070401819 shutdown_connections 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 -- 172.21.15.3:0/4070401819 wait complete. 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.316+0000 7fc766bc5700 1 Processor -- start 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc766bc5700 1 -- start start 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc766bc5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc76019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc766bc5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc76010d390 con 0x7fc760108de0 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc764961700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc76019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc764961700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc76019cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57804/0 (socket says 172.21.15.3:57804) 2024-05-08T05:42:01.098 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc764961700 1 -- 172.21.15.3:0/8505228 learned_addr learned my addr 172.21.15.3:0/8505228 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc764961700 1 -- 172.21.15.3:0/8505228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76019d290 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc764961700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc76019cd50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc754000c00 tx=0x7fc754011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc754003b20 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.317+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc75401a8c0 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.318+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc754024940 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.318+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76019d520 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.318+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7601a0210 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.319+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fc754022020 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.319+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc760064b00 con 0x7fc760108de0 2024-05-08T05:42:01.099 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.319+0000 7fc75dffb700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fc75003e470 0x7fc750040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.319+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc7540533a0 con 0x7fc760108de0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.319+0000 7fc75ffff700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fc75003e470 0x7fc750040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.319+0000 7fc75ffff700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fc75003e470 0x7fc750040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc74c006fd0 tx=0x7fc74c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.324+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc754053100 con 0x7fc760108de0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.598+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc7601a03e0 con 0x7fc760108de0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.650+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc7601a03e0 con 0x7fc760108de0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.652+0000 7fc75dffb700 1 -- 172.21.15.3:0/8505228 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fc75401aa70 con 0x7fc760108de0 2024-05-08T05:42:01.100 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fc75003e470 msgr2=0x7fc750040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fc75003e470 0x7fc750040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc74c006fd0 tx=0x7fc74c006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 msgr2=0x7fc76019cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc76019cd50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc754000c00 tx=0x7fc754011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 shutdown_connections 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fc75003e470 0x7fc750040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 --2- 172.21.15.3:0/8505228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc760108de0 0x7fc76019cd50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 >> 172.21.15.3:0/8505228 conn(0x7fc7601043a0 msgr2=0x7fc760104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 shutdown_connections 2024-05-08T05:42:01.101 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:41:58.653+0000 7fc766bc5700 1 -- 172.21.15.3:0/8505228 wait complete. 2024-05-08T05:42:01.340 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:01.168+0000 7f6af9e7a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:42:01.597 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:01.474+0000 7f6af9e7a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:42:01.598 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:01.574+0000 7f6af9e7a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:42:01.863 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:01.790+0000 7f6af9e7a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:42:02.117 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:01 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1900181617' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:42:02.117 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:01.996+0000 7f6af9e7a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:02.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:02.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T05:42:02.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:42:02.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:42:02.343 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.575+0000 7f88607a0700 1 Processor -- start 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f88607a0700 1 -- start start 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f88607a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f88581080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f88607a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8858108680 con 0x7f8858107c90 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f885e53c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f88581080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f885e53c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f88581080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57824/0 (socket says 172.21.15.3:57824) 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f885e53c700 1 -- 172.21.15.3:0/467258058 learned_addr learned my addr 172.21.15.3:0/467258058 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.576+0000 7f885e53c700 1 -- 172.21.15.3:0/467258058 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858108e90 con 0x7f8858107c90 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f885e53c700 1 --2- 172.21.15.3:0/467258058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f88581080b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f88540098a0 tx=0x7f8854009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33e2d0e010aeb47 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f885d53a700 1 -- 172.21.15.3:0/467258058 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854004160 con 0x7f8858107c90 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f885d53a700 1 -- 172.21.15.3:0/467258058 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8854004300 con 0x7f8858107c90 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f885d53a700 1 -- 172.21.15.3:0/467258058 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854011af0 con 0x7f8858107c90 2024-05-08T05:42:02.344 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f88607a0700 1 -- 172.21.15.3:0/467258058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 msgr2=0x7f88581080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f88607a0700 1 --2- 172.21.15.3:0/467258058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f88581080b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f88540098a0 tx=0x7f8854009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f88607a0700 1 -- 172.21.15.3:0/467258058 shutdown_connections 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f88607a0700 1 --2- 172.21.15.3:0/467258058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f88581080b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.577+0000 7f88607a0700 1 -- 172.21.15.3:0/467258058 >> 172.21.15.3:0/467258058 conn(0x7f8858103210 msgr2=0x7f8858105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f88607a0700 1 -- 172.21.15.3:0/467258058 shutdown_connections 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f88607a0700 1 -- 172.21.15.3:0/467258058 wait complete. 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f88607a0700 1 Processor -- start 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f88607a0700 1 -- start start 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f88607a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f8858192e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f88607a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f885810c240 con 0x7f8858107c90 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f885e53c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f8858192e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f885e53c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f8858192e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57834/0 (socket says 172.21.15.3:57834) 2024-05-08T05:42:02.345 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.578+0000 7f885e53c700 1 -- 172.21.15.3:0/1900181617 learned_addr learned my addr 172.21.15.3:0/1900181617 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f885e53c700 1 -- 172.21.15.3:0/1900181617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858193340 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f885e53c700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f8858192e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8854000c00 tx=0x7f8854011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854003b20 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f885401a8c0 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8858191530 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8854024940 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.579+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8858191a70 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.580+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8858064b00 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.580+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f8854022020 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.581+0000 7f884f7fe700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 0x7f8844040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.581+0000 7f885dd3b700 1 -- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 msgr2=0x7f8844040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.581+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8854053410 con 0x7f8858107c90 2024-05-08T05:42:02.346 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.581+0000 7f885dd3b700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 0x7f8844040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.586+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88540042a0 con 0x7f8858107c90 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.782+0000 7f885dd3b700 1 -- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 msgr2=0x7f8844040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.782+0000 7f885dd3b700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 0x7f8844040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.862+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8858196b80 con 0x7f8858107c90 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.863+0000 7f884f7fe700 1 -- 172.21.15.3:0/1900181617 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f88540212c0 con 0x7f8858107c90 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.865+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 msgr2=0x7f8844040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.865+0000 7f88607a0700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 0x7f8844040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.865+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 msgr2=0x7f8858192e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.865+0000 7f88607a0700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f8858192e00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8854000c00 tx=0x7f8854011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.866+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 shutdown_connections 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.866+0000 7f88607a0700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7f884403e470 0x7f8844040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.866+0000 7f88607a0700 1 --2- 172.21.15.3:0/1900181617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8858107c90 0x7f8858192e00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.866+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 >> 172.21.15.3:0/1900181617 conn(0x7f8858103210 msgr2=0x7f88581040f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:02.347 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.866+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 shutdown_connections 2024-05-08T05:42:02.348 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:01.866+0000 7f88607a0700 1 -- 172.21.15.3:0/1900181617 wait complete. 2024-05-08T05:42:02.348 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-05-08T05:42:02.348 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 5... 2024-05-08T05:42:02.753 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:02.644+0000 7f6af9e7a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:42:03.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:02.888+0000 7f6af9e7a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:42:03.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:03.331+0000 7f6af9e7a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:42:03.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:03.432+0000 7f6af9e7a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:42:04.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:03.817+0000 7f6af9e7a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:42:04.544 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:04.192+0000 7f6af9e7a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:42:04.544 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:04.289+0000 7f6af9e7a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:42:04.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:04.544+0000 7f6af9e7a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:42:04.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:04.684+0000 7f6af9e7a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:42:04.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:04.784+0000 7f6af9e7a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:42:05.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:05.122+0000 7f6af9e7a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:05.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:05.219+0000 7f6af9e7a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:42:06.080 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:05.852+0000 7f6af9e7a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:42:06.080 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:06 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:06.079+0000 7f6af9e7a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: Active manager daemon x restarted 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: Activating manager daemon x 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: mgrmap e6: x(active, starting, since 0.00354078s) 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:42:06.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: Manager daemon x is now available 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:06.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:06 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: Found migration_current of "None". Setting to last migration. 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: [08/May/2024:05:42:06] ENGINE Bus STARTING 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: [08/May/2024:05:42:06] ENGINE Serving on http://172.21.15.3:8765 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:07.407 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:07 smithi003 ceph-mon[30137]: mgrmap e7: x(active, since 1.00743s) 2024-05-08T05:42:07.700 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.780+0000 7fdd91d14700 1 Processor -- start 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.780+0000 7fdd91d14700 1 -- start start 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd91d14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd91d14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c1097d0 con 0x7fdd8c108de0 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd8b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd8b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35284/0 (socket says 172.21.15.3:35284) 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd8b7fe700 1 -- 172.21.15.3:0/2685234063 learned_addr learned my addr 172.21.15.3:0/2685234063 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd8b7fe700 1 -- 172.21.15.3:0/2685234063 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c109fe0 con 0x7fdd8c108de0 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd8b7fe700 1 --2- 172.21.15.3:0/2685234063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c109200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd740098a0 tx=0x7fdd74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2393feb1041cd9f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.781+0000 7fdd8a7fc700 1 -- 172.21.15.3:0/2685234063 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd74004160 con 0x7fdd8c108de0 2024-05-08T05:42:07.701 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd8a7fc700 1 -- 172.21.15.3:0/2685234063 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdd74004300 con 0x7fdd8c108de0 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd8a7fc700 1 -- 172.21.15.3:0/2685234063 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd74011af0 con 0x7fdd8c108de0 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 -- 172.21.15.3:0/2685234063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 msgr2=0x7fdd8c109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 --2- 172.21.15.3:0/2685234063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c109200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdd740098a0 tx=0x7fdd74009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 -- 172.21.15.3:0/2685234063 shutdown_connections 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 --2- 172.21.15.3:0/2685234063 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c109200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 -- 172.21.15.3:0/2685234063 >> 172.21.15.3:0/2685234063 conn(0x7fdd8c1043a0 msgr2=0x7fdd8c1067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 -- 172.21.15.3:0/2685234063 shutdown_connections 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.782+0000 7fdd91d14700 1 -- 172.21.15.3:0/2685234063 wait complete. 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd91d14700 1 Processor -- start 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd91d14700 1 -- start start 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd91d14700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd91d14700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10d390 con 0x7fdd8c108de0 2024-05-08T05:42:07.702 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd8b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd8b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35298/0 (socket says 172.21.15.3:35298) 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.783+0000 7fdd8b7fe700 1 -- 172.21.15.3:0/65572575 learned_addr learned my addr 172.21.15.3:0/65572575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd8b7fe700 1 -- 172.21.15.3:0/65572575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c19d290 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd8b7fe700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c19cd50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd74000c00 tx=0x7fdd74011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd74003b20 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c19d520 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdd7401a8c0 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdd74024940 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.784+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c1a0210 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.785+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fdd74022020 con 0x7fdd8c108de0 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.786+0000 7fdd88ff9700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.786+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 --> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd78041040 con 0x7fdd7803e470 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.786+0000 7fdd8affd700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 msgr2=0x7fdd78040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:07.703 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.786+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdd74053580 con 0x7fdd8c108de0 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.786+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.987+0000 7fdd8affd700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 msgr2=0x7fdd78040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:02.987+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.388+0000 7fdd8affd700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 msgr2=0x7fdd78040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:03.388+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:04.189+0000 7fdd8affd700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 msgr2=0x7fdd78040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:04.189+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:05.790+0000 7fdd8affd700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 msgr2=0x7fdd78040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/3412581798 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:05.790+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.083+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7fdd7401aae0 con 0x7fdd8c108de0 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.083+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 msgr2=0x7fdd78040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:06.083+0000 7fdd88ff9700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.085+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7fdd740551a0 con 0x7fdd8c108de0 2024-05-08T05:42:07.704 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.085+0000 7fdd88ff9700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fdd78041d60 0x7fdd78044150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.085+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdd74025e70 con 0x7fdd78041d60 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.086+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fdd78041d60 0x7fdd78044150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.087+0000 7fdd8affd700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fdd78041d60 0x7fdd78044150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c003a10 tx=0x7fdd7c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.087+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fdd74025e70 con 0x7fdd78041d60 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.096+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fdd8c06a5b0 con 0x7fdd78041d60 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd88ff9700 1 -- 172.21.15.3:0/65572575 <== mgr.14118 v2:172.21.15.3:6800/174548513 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fdd8c06a5b0 con 0x7fdd78041d60 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fdd78041d60 msgr2=0x7fdd78044150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fdd78041d60 0x7fdd78044150 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c003a10 tx=0x7fdd7c009290 comp rx=0 tx=0).stop 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 msgr2=0x7fdd8c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c19cd50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdd74000c00 tx=0x7fdd74011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 shutdown_connections 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fdd78041d60 0x7fdd78044150 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.705 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdd8c108de0 0x7fdd8c19cd50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.706 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 --2- 172.21.15.3:0/65572575 >> [v2:172.21.15.3:6800/3412581798,v1:172.21.15.3:6801/3412581798] conn(0x7fdd7803e470 0x7fdd78040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:07.706 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 >> 172.21.15.3:0/65572575 conn(0x7fdd8c1043a0 msgr2=0x7fdd8c104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:07.706 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 shutdown_connections 2024-05-08T05:42:07.706 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:07.097+0000 7fdd91d14700 1 -- 172.21.15.3:0/65572575 wait complete. 2024-05-08T05:42:07.706 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 5 is available 2024-05-08T05:42:07.706 INFO:teuthology.orchestra.run.smithi003.stdout:Setting orchestrator backend to cephadm... 2024-05-08T05:42:08.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:08 smithi003 ceph-mon[30137]: [08/May/2024:05:42:06] ENGINE Serving on https://172.21.15.3:7150 2024-05-08T05:42:08.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:08 smithi003 ceph-mon[30137]: [08/May/2024:05:42:06] ENGINE Bus STARTED 2024-05-08T05:42:08.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:08 smithi003 ceph-mon[30137]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:42:08.349 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:08 smithi003 ceph-mon[30137]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:42:09.229 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.105+0000 7f8620e1a700 1 Processor -- start 2024-05-08T05:42:09.229 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.105+0000 7f8620e1a700 1 -- start start 2024-05-08T05:42:09.229 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f8620e1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c10bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f8620e1a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f861c07d6e0 con 0x7f861c109840 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c10bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c10bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35370/0 (socket says 172.21.15.3:35370) 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861a59c700 1 -- 172.21.15.3:0/2653750074 learned_addr learned my addr 172.21.15.3:0/2653750074 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861a59c700 1 -- 172.21.15.3:0/2653750074 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c10c170 con 0x7f861c109840 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861a59c700 1 --2- 172.21.15.3:0/2653750074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c10bc30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f860c009e80 tx=0x7f860c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166faa70a07f2eaf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861959a700 1 -- 172.21.15.3:0/2653750074 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f860c017070 con 0x7f861c109840 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861959a700 1 -- 172.21.15.3:0/2653750074 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f860c00b8c0 con 0x7f861c109840 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.106+0000 7f861959a700 1 -- 172.21.15.3:0/2653750074 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f860c004490 con 0x7f861c109840 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 -- 172.21.15.3:0/2653750074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 msgr2=0x7f861c10bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 --2- 172.21.15.3:0/2653750074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c10bc30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f860c009e80 tx=0x7f860c00b330 comp rx=0 tx=0).stop 2024-05-08T05:42:09.230 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 -- 172.21.15.3:0/2653750074 shutdown_connections 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 --2- 172.21.15.3:0/2653750074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c10bc30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 -- 172.21.15.3:0/2653750074 >> 172.21.15.3:0/2653750074 conn(0x7f861c103210 msgr2=0x7f861c105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 -- 172.21.15.3:0/2653750074 shutdown_connections 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.107+0000 7f8620e1a700 1 -- 172.21.15.3:0/2653750074 wait complete. 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f8620e1a700 1 Processor -- start 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f8620e1a700 1 -- start start 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f8620e1a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f8620e1a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f861c10d130 con 0x7f861c109840 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f861a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f861a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35378/0 (socket says 172.21.15.3:35378) 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.108+0000 7f861a59c700 1 -- 172.21.15.3:0/860337824 learned_addr learned my addr 172.21.15.3:0/860337824 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f861a59c700 1 -- 172.21.15.3:0/860337824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861c19d1b0 con 0x7f861c109840 2024-05-08T05:42:09.231 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f861a59c700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c19cc70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f860c00b7b0 tx=0x7f860c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f860c020020 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f860c012430 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861c19d440 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f860c017040 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.109+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f861c1a0130 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.110+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f861c064b00 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.110+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f860c023020 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.111+0000 7f86137fe700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f860803e360 0x7f8608040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.111+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f860c00f050 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.111+0000 7f8619d9b700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f860803e360 0x7f8608040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.111+0000 7f8619d9b700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f860803e360 0x7f8608040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8604006fd0 tx=0x7f8604006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.116+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f860c02a020 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.136+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f860c011d40 con 0x7f861c109840 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.348+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f861c1043c0 con 0x7f860803e360 2024-05-08T05:42:09.232 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.353+0000 7f86137fe700 1 -- 172.21.15.3:0/860337824 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f861c1043c0 con 0x7f860803e360 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f860803e360 msgr2=0x7f8608040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f860803e360 0x7f8608040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8604006fd0 tx=0x7f8604006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 msgr2=0x7f861c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c19cc70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f860c00b7b0 tx=0x7f860c00be20 comp rx=0 tx=0).stop 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 shutdown_connections 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f860803e360 0x7f8608040820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 --2- 172.21.15.3:0/860337824 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f861c109840 0x7f861c19cc70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 >> 172.21.15.3:0/860337824 conn(0x7f861c103210 msgr2=0x7f861c103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 shutdown_connections 2024-05-08T05:42:09.233 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:08.356+0000 7f8620e1a700 1 -- 172.21.15.3:0/860337824 wait complete. 2024-05-08T05:42:09.338 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:09 smithi003 ceph-mon[30137]: mgrmap e8: x(active, since 2s) 2024-05-08T05:42:09.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:09 smithi003 ceph-mon[30137]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:09.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:09 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:09.339 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:09 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:10.335 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T05:42:10.335 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.643+0000 7f659b599700 1 Processor -- start 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.643+0000 7f659b599700 1 -- start start 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f659b599700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f6594109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f659b599700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f65941097d0 con 0x7f6594108de0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f6599335700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f6594109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f6599335700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f6594109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35380/0 (socket says 172.21.15.3:35380) 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f6599335700 1 -- 172.21.15.3:0/2217564950 learned_addr learned my addr 172.21.15.3:0/2217564950 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f6599335700 1 -- 172.21.15.3:0/2217564950 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6594109fe0 con 0x7f6594108de0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.644+0000 7f6599335700 1 --2- 172.21.15.3:0/2217564950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f6594109200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65880098a0 tx=0x7f6588009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109e785f6567fcce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f6587fff700 1 -- 172.21.15.3:0/2217564950 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6588004160 con 0x7f6594108de0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f6587fff700 1 -- 172.21.15.3:0/2217564950 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6588004300 con 0x7f6594108de0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f6587fff700 1 -- 172.21.15.3:0/2217564950 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6588011af0 con 0x7f6594108de0 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 -- 172.21.15.3:0/2217564950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 msgr2=0x7f6594109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.336 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 --2- 172.21.15.3:0/2217564950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f6594109200 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f65880098a0 tx=0x7f6588009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 -- 172.21.15.3:0/2217564950 shutdown_connections 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 --2- 172.21.15.3:0/2217564950 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f6594109200 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 -- 172.21.15.3:0/2217564950 >> 172.21.15.3:0/2217564950 conn(0x7f65941043a0 msgr2=0x7f65941067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 -- 172.21.15.3:0/2217564950 shutdown_connections 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.645+0000 7f659b599700 1 -- 172.21.15.3:0/2217564950 wait complete. 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f659b599700 1 Processor -- start 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f659b599700 1 -- start start 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f659b599700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f659419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f659b599700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f659410d390 con 0x7f6594108de0 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f6599335700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f659419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f6599335700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f659419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35390/0 (socket says 172.21.15.3:35390) 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.646+0000 7f6599335700 1 -- 172.21.15.3:0/4006952796 learned_addr learned my addr 172.21.15.3:0/4006952796 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f6599335700 1 -- 172.21.15.3:0/4006952796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659419d290 con 0x7f6594108de0 2024-05-08T05:42:10.337 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f6599335700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f659419cd50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6588000c00 tx=0x7f6588011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6588003b20 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f658801a8c0 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659419d520 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6588024940 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.647+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65941a0210 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.648+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659406a5b0 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.648+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f6588022020 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.648+0000 7f65867fc700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f658003e4c0 0x7f6580040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.648+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6588053790 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.648+0000 7f6598b34700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f658003e4c0 0x7f6580040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.649+0000 7f6598b34700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f658003e4c0 0x7f6580040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6590006fd0 tx=0x7f6590006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.654+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f658801aa70 con 0x7f6594108de0 2024-05-08T05:42:10.338 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.879+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6594105610 con 0x7f658003e4c0 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.879+0000 7f65867fc700 1 -- 172.21.15.3:0/4006952796 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6594105610 con 0x7f658003e4c0 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f658003e4c0 msgr2=0x7f6580040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f658003e4c0 0x7f6580040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6590006fd0 tx=0x7f6590006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 msgr2=0x7f659419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f659419cd50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6588000c00 tx=0x7f6588011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 shutdown_connections 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f658003e4c0 0x7f6580040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 --2- 172.21.15.3:0/4006952796 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6594108de0 0x7f659419cd50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 >> 172.21.15.3:0/4006952796 conn(0x7f65941043a0 msgr2=0x7f6594104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 shutdown_connections 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:09.882+0000 7f659b599700 1 -- 172.21.15.3:0/4006952796 wait complete. 2024-05-08T05:42:10.339 INFO:teuthology.orchestra.run.smithi003.stdout:Generating ssh key... 2024-05-08T05:42:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:10 smithi003 ceph-mon[30137]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:11.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: Generating public/private rsa key pair. 2024-05-08T05:42:11.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: Your identification has been saved in /tmp/tmpd2nx8uj7/key. 2024-05-08T05:42:11.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: Your public key has been saved in /tmp/tmpd2nx8uj7/key.pub. 2024-05-08T05:42:11.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: The key fingerprint is: 2024-05-08T05:42:11.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: SHA256:D0dMDMJJHbJIRMVuL4GHZAVa+gtr2AyAo4Fool7RFKE ceph-708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:11.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: The key's randomart image is: 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: +---[RSA 3072]----+ 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: | o*@Ooo+. | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: |+ =*.++.o. | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: |O.oE.=. o | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: |*o .+ = . | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: |+ ...o oS . | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: |.=.o .. .+ | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: |..= . . . | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: | . | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: | | 2024-05-08T05:42:11.411 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: +----[SHA256]-----+ 2024-05-08T05:42:11.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.702+0000 7f7a74b69700 1 Processor -- start 2024-05-08T05:42:11.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.702+0000 7f7a74b69700 1 -- start start 2024-05-08T05:42:11.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.702+0000 7f7a74b69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a70109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.702+0000 7f7a74b69700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70109d00 con 0x7f7a70109310 2024-05-08T05:42:11.719 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a70109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a70109730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35400/0 (socket says 172.21.15.3:35400) 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6e59c700 1 -- 172.21.15.3:0/2164022051 learned_addr learned my addr 172.21.15.3:0/2164022051 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6e59c700 1 -- 172.21.15.3:0/2164022051 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a70107ee0 con 0x7f7a70109310 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6e59c700 1 --2- 172.21.15.3:0/2164022051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a70109730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098a0 tx=0x7f7a60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c015715bff37ec77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6d59a700 1 -- 172.21.15.3:0/2164022051 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a60004160 con 0x7f7a70109310 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6d59a700 1 -- 172.21.15.3:0/2164022051 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a60004300 con 0x7f7a70109310 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.703+0000 7f7a6d59a700 1 -- 172.21.15.3:0/2164022051 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a60011af0 con 0x7f7a70109310 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 -- 172.21.15.3:0/2164022051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 msgr2=0x7f7a70109730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 --2- 172.21.15.3:0/2164022051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a70109730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a600098a0 tx=0x7f7a60009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 -- 172.21.15.3:0/2164022051 shutdown_connections 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 --2- 172.21.15.3:0/2164022051 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a70109730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 -- 172.21.15.3:0/2164022051 >> 172.21.15.3:0/2164022051 conn(0x7f7a701029d0 msgr2=0x7f7a70104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:11.720 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 -- 172.21.15.3:0/2164022051 shutdown_connections 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.704+0000 7f7a74b69700 1 -- 172.21.15.3:0/2164022051 wait complete. 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a74b69700 1 Processor -- start 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a74b69700 1 -- start start 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a74b69700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a7019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a74b69700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7010c2a0 con 0x7f7a70109310 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a6e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a7019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a6e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a7019ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35408/0 (socket says 172.21.15.3:35408) 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a6e59c700 1 -- 172.21.15.3:0/1752766845 learned_addr learned my addr 172.21.15.3:0/1752766845 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.705+0000 7f7a6e59c700 1 -- 172.21.15.3:0/1752766845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7019d340 con 0x7f7a70109310 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.706+0000 7f7a6e59c700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a7019ce00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a60011ac0 tx=0x7f7a60011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.706+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a60003970 con 0x7f7a70109310 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.706+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a60003b10 con 0x7f7a70109310 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.706+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a60024580 con 0x7f7a70109310 2024-05-08T05:42:11.721 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.706+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7019d5d0 con 0x7f7a70109310 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.706+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a0270 con 0x7f7a70109310 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.707+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7a6001a430 con 0x7f7a70109310 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.707+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a70064b00 con 0x7f7a70109310 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.707+0000 7f7a5f7fe700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7a5803e4c0 0x7f7a58040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.707+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a600536e0 con 0x7f7a70109310 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.707+0000 7f7a6dd9b700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7a5803e4c0 0x7f7a58040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.708+0000 7f7a6dd9b700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7a5803e4c0 0x7f7a58040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006fd0 tx=0x7f7a64006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.712+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a60005a20 con 0x7f7a70109310 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:10.926+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7a700008d0 con 0x7f7a5803e4c0 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.119+0000 7f7a5f7fe700 1 -- 172.21.15.3:0/1752766845 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7a700008d0 con 0x7f7a5803e4c0 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7a5803e4c0 msgr2=0x7f7a58040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7a5803e4c0 0x7f7a58040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006fd0 tx=0x7f7a64006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:11.722 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 msgr2=0x7f7a7019ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a7019ce00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a60011ac0 tx=0x7f7a60011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 shutdown_connections 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7a5803e4c0 0x7f7a58040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 --2- 172.21.15.3:0/1752766845 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7a70109310 0x7f7a7019ce00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 >> 172.21.15.3:0/1752766845 conn(0x7f7a701029d0 msgr2=0x7f7a70103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 shutdown_connections 2024-05-08T05:42:11.723 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:11.122+0000 7f7a74b69700 1 -- 172.21.15.3:0/1752766845 wait complete. 2024-05-08T05:42:12.347 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:12 smithi003 ceph-mon[30137]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:12.347 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:12 smithi003 ceph-mon[30137]: Generating ssh key... 2024-05-08T05:42:12.347 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:12 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:12.347 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:12 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:12.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:12.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.114+0000 7f5209ec1700 1 Processor -- start 2024-05-08T05:42:12.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.114+0000 7f5209ec1700 1 -- start start 2024-05-08T05:42:12.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.114+0000 7f5209ec1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f52041080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.858 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.114+0000 7f5209ec1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52041086c0 con 0x7f5204107cd0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.114+0000 7f52037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f52041080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f52037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f52041080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35424/0 (socket says 172.21.15.3:35424) 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f52037fe700 1 -- 172.21.15.3:0/673507617 learned_addr learned my addr 172.21.15.3:0/673507617 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f52037fe700 1 -- 172.21.15.3:0/673507617 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5204108ed0 con 0x7f5204107cd0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f52037fe700 1 --2- 172.21.15.3:0/673507617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f52041080f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098a0 tx=0x7f51f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c61251baa4af86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f5202ffd700 1 -- 172.21.15.3:0/673507617 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4004160 con 0x7f5204107cd0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f5202ffd700 1 -- 172.21.15.3:0/673507617 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51f4004300 con 0x7f5204107cd0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.115+0000 7f5202ffd700 1 -- 172.21.15.3:0/673507617 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4011af0 con 0x7f5204107cd0 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 -- 172.21.15.3:0/673507617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 msgr2=0x7f52041080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 --2- 172.21.15.3:0/673507617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f52041080f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f51f40098a0 tx=0x7f51f4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 -- 172.21.15.3:0/673507617 shutdown_connections 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 --2- 172.21.15.3:0/673507617 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f52041080f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 -- 172.21.15.3:0/673507617 >> 172.21.15.3:0/673507617 conn(0x7f52041032b0 msgr2=0x7f52041056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 -- 172.21.15.3:0/673507617 shutdown_connections 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.116+0000 7f5209ec1700 1 -- 172.21.15.3:0/673507617 wait complete. 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f5209ec1700 1 Processor -- start 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f5209ec1700 1 -- start start 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f5209ec1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f520419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f5209ec1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f520410c280 con 0x7f5204107cd0 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f52037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f520419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f52037fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f520419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35434/0 (socket says 172.21.15.3:35434) 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f52037fe700 1 -- 172.21.15.3:0/308477747 learned_addr learned my addr 172.21.15.3:0/308477747 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:12.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.117+0000 7f52037fe700 1 -- 172.21.15.3:0/308477747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520419d280 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.118+0000 7f52037fe700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f520419cd40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.118+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4003b20 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.118+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f51f401a8c0 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.118+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520419d510 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.118+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51f4024940 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.118+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041a0200 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.119+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f51f4022020 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.119+0000 7f52017fa700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f51f003e4c0 0x7f51f0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.119+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f51f40546d0 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.119+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5204064b00 con 0x7f5204107cd0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.119+0000 7f51faffd700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f51f003e4c0 0x7f51f0040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.120+0000 7f51faffd700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f51f003e4c0 0x7f51f0040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51ec006fd0 tx=0x7f51ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:12.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.125+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51f4029df0 con 0x7f5204107cd0 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.347+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f520402e7d0 con 0x7f51f003e4c0 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.347+0000 7f52017fa700 1 -- 172.21.15.3:0/308477747 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f520402e7d0 con 0x7f51f003e4c0 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f51f003e4c0 msgr2=0x7f51f0040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f51f003e4c0 0x7f51f0040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51ec006fd0 tx=0x7f51ec006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 msgr2=0x7f520419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f520419cd40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f51f4000c00 tx=0x7f51f4011b70 comp rx=0 tx=0).stop 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 shutdown_connections 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f51f003e4c0 0x7f51f0040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 --2- 172.21.15.3:0/308477747 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5204107cd0 0x7f520419cd40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 >> 172.21.15.3:0/308477747 conn(0x7f52041032b0 msgr2=0x7f5204104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 shutdown_connections 2024-05-08T05:42:12.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:12.350+0000 7f5209ec1700 1 -- 172.21.15.3:0/308477747 wait complete. 2024-05-08T05:42:12.863 INFO:teuthology.orchestra.run.smithi003.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:42:12.863 INFO:teuthology.orchestra.run.smithi003.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T05:42:12.863 INFO:teuthology.orchestra.run.smithi003.stdout:Adding host smithi003... 2024-05-08T05:42:13.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:13 smithi003 ceph-mon[30137]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:14.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:14 smithi003 ceph-mon[30137]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:15.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:15 smithi003 ceph-mon[30137]: Deploying cephadm binary to smithi003 2024-05-08T05:42:18.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Added host 'smithi003' with addr '172.21.15.3' 2024-05-08T05:42:18.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.265+0000 7fe8fedd4700 1 Processor -- start 2024-05-08T05:42:18.248 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.265+0000 7fe8fedd4700 1 -- start start 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fedd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f81080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fedd4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8108680 con 0x7fe8f8107c90 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fcb70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f81080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fcb70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f81080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55098/0 (socket says 172.21.15.3:55098) 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fcb70700 1 -- 172.21.15.3:0/504866463 learned_addr learned my addr 172.21.15.3:0/504866463 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fcb70700 1 -- 172.21.15.3:0/504866463 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f8108e90 con 0x7fe8f8107c90 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8fcb70700 1 --2- 172.21.15.3:0/504866463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f81080b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009e80 tx=0x7fe8f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f63c453804ed2ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8ef7fe700 1 -- 172.21.15.3:0/504866463 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4017070 con 0x7fe8f8107c90 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8ef7fe700 1 -- 172.21.15.3:0/504866463 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe8f400b8c0 con 0x7fe8f8107c90 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.266+0000 7fe8ef7fe700 1 -- 172.21.15.3:0/504866463 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4004490 con 0x7fe8f8107c90 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 -- 172.21.15.3:0/504866463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 msgr2=0x7fe8f81080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 --2- 172.21.15.3:0/504866463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f81080b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4009e80 tx=0x7fe8f400b330 comp rx=0 tx=0).stop 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 -- 172.21.15.3:0/504866463 shutdown_connections 2024-05-08T05:42:18.249 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 --2- 172.21.15.3:0/504866463 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f81080b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 -- 172.21.15.3:0/504866463 >> 172.21.15.3:0/504866463 conn(0x7fe8f8103250 msgr2=0x7fe8f8105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 -- 172.21.15.3:0/504866463 shutdown_connections 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.267+0000 7fe8fedd4700 1 -- 172.21.15.3:0/504866463 wait complete. 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fedd4700 1 Processor -- start 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fedd4700 1 -- start start 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fedd4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fedd4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f810c240 con 0x7fe8f8107c90 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fcb70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fcb70700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55108/0 (socket says 172.21.15.3:55108) 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fcb70700 1 -- 172.21.15.3:0/1422826487 learned_addr learned my addr 172.21.15.3:0/1422826487 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fcb70700 1 -- 172.21.15.3:0/1422826487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f819d330 con 0x7fe8f8107c90 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.268+0000 7fe8fcb70700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f819cdf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400b7b0 tx=0x7fe8f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.250 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.269+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4020020 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.269+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe8f4012430 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.269+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4017040 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.269+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f819d5c0 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.269+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f81a02b0 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.270+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe8f4023020 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.270+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f8064b00 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.270+0000 7fe8edffb700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fe8e003e470 0x7fe8e0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.270+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe8f400f050 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.270+0000 7fe8effff700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fe8e003e470 0x7fe8e0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.271+0000 7fe8effff700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fe8e003e470 0x7fe8e0040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.275+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8f402a020 con 0x7fe8f8107c90 2024-05-08T05:42:18.251 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:13.495+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi003", "addr": "172.21.15.3", "target": ["mon-mgr", ""]}) -- 0x7fe8f802e8f0 con 0x7fe8e003e470 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.802+0000 7fe8edffb700 1 -- 172.21.15.3:0/1422826487 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fe8f802e8f0 con 0x7fe8e003e470 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fe8e003e470 msgr2=0x7fe8e0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fe8e003e470 0x7fe8e0040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8006fd0 tx=0x7fe8e8006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 msgr2=0x7fe8f819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f819cdf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400b7b0 tx=0x7fe8f400be20 comp rx=0 tx=0).stop 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 shutdown_connections 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fe8e003e470 0x7fe8e0040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 --2- 172.21.15.3:0/1422826487 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe8f8107c90 0x7fe8f819cdf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 >> 172.21.15.3:0/1422826487 conn(0x7fe8f8103250 msgr2=0x7fe8f8104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 shutdown_connections 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:17.805+0000 7fe8fedd4700 1 -- 172.21.15.3:0/1422826487 wait complete. 2024-05-08T05:42:18.252 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mon service... 2024-05-08T05:42:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:18 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:18 smithi003 ceph-mon[30137]: Added host smithi003 2024-05-08T05:42:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:18 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.701+0000 7f50c23ee700 1 Processor -- start 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.701+0000 7f50c23ee700 1 -- start start 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50c23ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc10bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50c23ee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc07d6a0 con 0x7f50bc109890 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc10bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc10bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55124/0 (socket says 172.21.15.3:55124) 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50bbfff700 1 -- 172.21.15.3:0/1442591406 learned_addr learned my addr 172.21.15.3:0/1442591406 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50bbfff700 1 -- 172.21.15.3:0/1442591406 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc10c1c0 con 0x7f50bc109890 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.702+0000 7f50bbfff700 1 --2- 172.21.15.3:0/1442591406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc10bc80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0098a0 tx=0x7f50ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582032d9f355a202 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50baffd700 1 -- 172.21.15.3:0/1442591406 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac004160 con 0x7f50bc109890 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50baffd700 1 -- 172.21.15.3:0/1442591406 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50ac004300 con 0x7f50bc109890 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50baffd700 1 -- 172.21.15.3:0/1442591406 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac011af0 con 0x7f50bc109890 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50c23ee700 1 -- 172.21.15.3:0/1442591406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 msgr2=0x7f50bc10bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50c23ee700 1 --2- 172.21.15.3:0/1442591406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc10bc80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f50ac0098a0 tx=0x7f50ac009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50c23ee700 1 -- 172.21.15.3:0/1442591406 shutdown_connections 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50c23ee700 1 --2- 172.21.15.3:0/1442591406 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc10bc80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.703+0000 7f50c23ee700 1 -- 172.21.15.3:0/1442591406 >> 172.21.15.3:0/1442591406 conn(0x7f50bc103220 msgr2=0x7f50bc105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.704+0000 7f50c23ee700 1 -- 172.21.15.3:0/1442591406 shutdown_connections 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.704+0000 7f50c23ee700 1 -- 172.21.15.3:0/1442591406 wait complete. 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.704+0000 7f50c23ee700 1 Processor -- start 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.704+0000 7f50c23ee700 1 -- start start 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50c23ee700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50c23ee700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc10d120 con 0x7f50bc109890 2024-05-08T05:42:19.600 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55128/0 (socket says 172.21.15.3:55128) 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50bbfff700 1 -- 172.21.15.3:0/1272994069 learned_addr learned my addr 172.21.15.3:0/1272994069 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50bbfff700 1 -- 172.21.15.3:0/1272994069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc19d290 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.705+0000 7f50bbfff700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc19cd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50ac011ac0 tx=0x7f50ac011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.706+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac003970 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.706+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50ac003b10 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.706+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc19d520 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.706+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50ac024610 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.706+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc1a0210 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.707+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50bc064b00 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.707+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f50ac01a9e0 con 0x7f50bc109890 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.707+0000 7f50b97fa700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f50a403e4c0 0x7f50a4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:19.601 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.707+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f50ac053800 con 0x7f50bc109890 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.707+0000 7f50bb7fe700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f50a403e4c0 0x7f50a4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.708+0000 7f50bb7fe700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f50a403e4c0 0x7f50a4040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50b0006fd0 tx=0x7f50b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.716+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f50ac055d00 con 0x7f50bc109890 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.954+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f50bc104500 con 0x7f50a403e4c0 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.959+0000 7f50b97fa700 1 -- 172.21.15.3:0/1272994069 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f50bc104500 con 0x7f50a403e4c0 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f50a403e4c0 msgr2=0x7f50a4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f50a403e4c0 0x7f50a4040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50b0006fd0 tx=0x7f50b0006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 msgr2=0x7f50bc19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc19cd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50ac011ac0 tx=0x7f50ac011ba0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 shutdown_connections 2024-05-08T05:42:19.602 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f50a403e4c0 0x7f50a4040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 --2- 172.21.15.3:0/1272994069 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50bc109890 0x7f50bc19cd50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:19.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 >> 172.21.15.3:0/1272994069 conn(0x7f50bc103220 msgr2=0x7f50bc103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:19.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 shutdown_connections 2024-05-08T05:42:19.603 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:18.962+0000 7f50c23ee700 1 -- 172.21.15.3:0/1272994069 wait complete. 2024-05-08T05:42:19.603 INFO:teuthology.orchestra.run.smithi003.stdout:Deploying unmanaged mgr service... 2024-05-08T05:42:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:19 smithi003 ceph-mon[30137]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:19 smithi003 ceph-mon[30137]: Saving service mon spec with placement count:5 2024-05-08T05:42:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:19 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:21.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:21 smithi003 ceph-mon[30137]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:21.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:21 smithi003 ceph-mon[30137]: Saving service mgr spec with placement count:2 2024-05-08T05:42:21.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:21 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:21.587 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:21 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:21.595 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.078+0000 7f8aa6748700 1 Processor -- start 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.078+0000 7f8aa6748700 1 -- start start 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8aa6748700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa010bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8aa6748700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa007d6e0 con 0x7f8aa0109890 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8a9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa010bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8a9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa010bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55134/0 (socket says 172.21.15.3:55134) 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8a9ffff700 1 -- 172.21.15.3:0/4073226982 learned_addr learned my addr 172.21.15.3:0/4073226982 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8a9ffff700 1 -- 172.21.15.3:0/4073226982 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa010c1c0 con 0x7f8aa0109890 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.079+0000 7f8a9ffff700 1 --2- 172.21.15.3:0/4073226982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa010bc80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8a900098a0 tx=0x7f8a90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be8ef8723e8a67c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8a9effd700 1 -- 172.21.15.3:0/4073226982 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a90004160 con 0x7f8aa0109890 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8a9effd700 1 -- 172.21.15.3:0/4073226982 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a90004300 con 0x7f8aa0109890 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8a9effd700 1 -- 172.21.15.3:0/4073226982 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a90011af0 con 0x7f8aa0109890 2024-05-08T05:42:21.596 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 -- 172.21.15.3:0/4073226982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 msgr2=0x7f8aa010bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 --2- 172.21.15.3:0/4073226982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa010bc80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8a900098a0 tx=0x7f8a90009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 -- 172.21.15.3:0/4073226982 shutdown_connections 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 --2- 172.21.15.3:0/4073226982 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa010bc80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 -- 172.21.15.3:0/4073226982 >> 172.21.15.3:0/4073226982 conn(0x7f8aa0103260 msgr2=0x7f8aa01056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 -- 172.21.15.3:0/4073226982 shutdown_connections 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.080+0000 7f8aa6748700 1 -- 172.21.15.3:0/4073226982 wait complete. 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8aa6748700 1 Processor -- start 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8aa6748700 1 -- start start 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8aa6748700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8aa6748700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa010d110 con 0x7f8aa0109890 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8a9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8a9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55148/0 (socket says 172.21.15.3:55148) 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.081+0000 7f8a9ffff700 1 -- 172.21.15.3:0/3808633067 learned_addr learned my addr 172.21.15.3:0/3808633067 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:21.597 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8a9ffff700 1 -- 172.21.15.3:0/3808633067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa019d1e0 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8a9ffff700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa019cca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8a90003f40 tx=0x7f8a90011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a900038b0 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8a90003a50 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a9001a5c0 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa019d470 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.082+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa01a0160 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.083+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f8a9001a870 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.083+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa0064b00 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.083+0000 7f8a9d7fa700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f8a8803e470 0x7f8a88040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.083+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8a90053a40 con 0x7f8aa0109890 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.083+0000 7f8a9f7fe700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f8a8803e470 0x7f8a88040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.084+0000 7f8a9f7fe700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f8a8803e470 0x7f8a88040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:21.598 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.089+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8a9000e260 con 0x7f8aa0109890 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.317+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8aa01044b0 con 0x7f8a8803e470 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.324+0000 7f8a9d7fa700 1 -- 172.21.15.3:0/3808633067 <== mgr.14118 v2:172.21.15.3:6800/174548513 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8aa01044b0 con 0x7f8a8803e470 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f8a8803e470 msgr2=0x7f8a88040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f8a8803e470 0x7f8a88040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 msgr2=0x7f8aa019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa019cca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8a90003f40 tx=0x7f8a90011890 comp rx=0 tx=0).stop 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 shutdown_connections 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f8a8803e470 0x7f8a88040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 --2- 172.21.15.3:0/3808633067 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8aa0109890 0x7f8aa019cca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 >> 172.21.15.3:0/3808633067 conn(0x7f8aa0103260 msgr2=0x7f8aa0103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 shutdown_connections 2024-05-08T05:42:21.599 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:20.326+0000 7f8aa6748700 1 -- 172.21.15.3:0/3808633067 wait complete. 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.020+0000 7fb09167f700 1 Processor -- start 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb09167f700 1 -- start start 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb09167f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c10bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb09167f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c07d6a0 con 0x7fb08c109890 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb08affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c10bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb08affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c10bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55150/0 (socket says 172.21.15.3:55150) 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb08affd700 1 -- 172.21.15.3:0/3281327503 learned_addr learned my addr 172.21.15.3:0/3281327503 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.021+0000 7fb08affd700 1 -- 172.21.15.3:0/3281327503 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c10c1c0 con 0x7fb08c109890 2024-05-08T05:42:22.859 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.022+0000 7fb08affd700 1 --2- 172.21.15.3:0/3281327503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c10bc80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0740098a0 tx=0x7fb074009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4c15cd7b29eb70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.022+0000 7fb089ffb700 1 -- 172.21.15.3:0/3281327503 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074004160 con 0x7fb08c109890 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.022+0000 7fb089ffb700 1 -- 172.21.15.3:0/3281327503 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb074004300 con 0x7fb08c109890 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.022+0000 7fb089ffb700 1 -- 172.21.15.3:0/3281327503 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074011a20 con 0x7fb08c109890 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 -- 172.21.15.3:0/3281327503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 msgr2=0x7fb08c10bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 --2- 172.21.15.3:0/3281327503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c10bc80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb0740098a0 tx=0x7fb074009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 -- 172.21.15.3:0/3281327503 shutdown_connections 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 --2- 172.21.15.3:0/3281327503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c10bc80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 -- 172.21.15.3:0/3281327503 >> 172.21.15.3:0/3281327503 conn(0x7fb08c103280 msgr2=0x7fb08c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 -- 172.21.15.3:0/3281327503 shutdown_connections 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 -- 172.21.15.3:0/3281327503 wait complete. 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.023+0000 7fb09167f700 1 Processor -- start 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb09167f700 1 -- start start 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb09167f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.860 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb09167f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb08c10d160 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb08affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb08affd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55152/0 (socket says 172.21.15.3:55152) 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb08affd700 1 -- 172.21.15.3:0/3908592719 learned_addr learned my addr 172.21.15.3:0/3908592719 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb08affd700 1 -- 172.21.15.3:0/3908592719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08c19d1c0 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb08affd700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c19cc80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0740116e0 tx=0x7fb074011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.024+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074003a60 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.025+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07401b8f0 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.025+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08c19d450 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.025+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb074019860 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.025+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08c1a0140 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.025+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08c064b00 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.026+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fb074019b60 con 0x7fb08c109890 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.026+0000 7fb083fff700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fb07803e420 0x7fb0780408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:22.861 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.026+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb074053ce0 con 0x7fb08c109890 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.026+0000 7fb08a7fc700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fb07803e420 0x7fb0780408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.026+0000 7fb08a7fc700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fb07803e420 0x7fb0780408e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006fd0 tx=0x7fb07c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.035+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb07401f030 con 0x7fb08c109890 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.250+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fb08c1a0710 con 0x7fb08c109890 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.255+0000 7fb083fff700 1 -- 172.21.15.3:0/3908592719 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fb0740223b0 con 0x7fb08c109890 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.257+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fb07803e420 msgr2=0x7fb0780408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.257+0000 7fb09167f700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fb07803e420 0x7fb0780408e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb07c006fd0 tx=0x7fb07c006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.257+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 msgr2=0x7fb08c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.257+0000 7fb09167f700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c19cc80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb0740116e0 tx=0x7fb074011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.258+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 shutdown_connections 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.258+0000 7fb09167f700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fb07803e420 0x7fb0780408e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.258+0000 7fb09167f700 1 --2- 172.21.15.3:0/3908592719 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb08c109890 0x7fb08c19cc80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.258+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 >> 172.21.15.3:0/3908592719 conn(0x7fb08c103280 msgr2=0x7fb08c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:22.862 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.258+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 shutdown_connections 2024-05-08T05:42:22.863 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:22.258+0000 7fb09167f700 1 -- 172.21.15.3:0/3908592719 wait complete. 2024-05-08T05:42:23.117 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:22 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:23.117 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:22 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3908592719' entity='client.admin' 2024-05-08T05:42:24.510 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f9195aa8700 1 Processor -- start 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f9195aa8700 1 -- start start 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f9195aa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f9190109660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f9195aa8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9190109c30 con 0x7f9190109240 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f918f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f9190109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f918f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f9190109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59734/0 (socket says 172.21.15.3:59734) 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.335+0000 7f918f7fe700 1 -- 172.21.15.3:0/158348814 learned_addr learned my addr 172.21.15.3:0/158348814 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.336+0000 7f918f7fe700 1 -- 172.21.15.3:0/158348814 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9190107ea0 con 0x7f9190109240 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.336+0000 7f918f7fe700 1 --2- 172.21.15.3:0/158348814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f9190109660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f918000bb40 tx=0x7f918000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdcba7a613cf1eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.336+0000 7f918e7fc700 1 -- 172.21.15.3:0/158348814 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f918000dbe0 con 0x7f9190109240 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.336+0000 7f918e7fc700 1 -- 172.21.15.3:0/158348814 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91800044e0 con 0x7f9190109240 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.336+0000 7f918e7fc700 1 -- 172.21.15.3:0/158348814 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9180010470 con 0x7f9190109240 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 -- 172.21.15.3:0/158348814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 msgr2=0x7f9190109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.511 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 --2- 172.21.15.3:0/158348814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f9190109660 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f918000bb40 tx=0x7f918000be50 comp rx=0 tx=0).stop 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 -- 172.21.15.3:0/158348814 shutdown_connections 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 --2- 172.21.15.3:0/158348814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f9190109660 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 -- 172.21.15.3:0/158348814 >> 172.21.15.3:0/158348814 conn(0x7f91901029d0 msgr2=0x7f9190104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 -- 172.21.15.3:0/158348814 shutdown_connections 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.337+0000 7f9195aa8700 1 -- 172.21.15.3:0/158348814 wait complete. 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f9195aa8700 1 Processor -- start 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f9195aa8700 1 -- start start 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f9195aa8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f919019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f9195aa8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9190106d90 con 0x7f9190109240 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f918f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f919019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f918f7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f919019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59746/0 (socket says 172.21.15.3:59746) 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.338+0000 7f918f7fe700 1 -- 172.21.15.3:0/2384696299 learned_addr learned my addr 172.21.15.3:0/2384696299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f918f7fe700 1 -- 172.21.15.3:0/2384696299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919019b050 con 0x7f9190109240 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f918f7fe700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f919019ab10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9180003d10 tx=0x7f9180003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.512 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9180027070 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9180010c20 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9180022c10 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919019b2e0 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.339+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919019dfd0 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.340+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f918002a070 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.340+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9190064b00 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.340+0000 7f918cff9700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f917c03e470 0x7f917c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.340+0000 7f918effd700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f917c03e470 0x7f917c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.340+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f918001f070 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.341+0000 7f918effd700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f917c03e470 0x7f917c040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9178006fd0 tx=0x7f9178006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.346+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9180015070 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.572+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f9190194070 con 0x7f9190109240 2024-05-08T05:42:24.513 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.575+0000 7f918cff9700 1 -- 172.21.15.3:0/2384696299 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f9190194070 con 0x7f9190109240 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f917c03e470 msgr2=0x7f917c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f917c03e470 0x7f917c040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9178006fd0 tx=0x7f9178006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 msgr2=0x7f919019ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f919019ab10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9180003d10 tx=0x7f9180003df0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 shutdown_connections 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f917c03e470 0x7f917c040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 --2- 172.21.15.3:0/2384696299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9190109240 0x7f919019ab10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.577+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 >> 172.21.15.3:0/2384696299 conn(0x7f91901029d0 msgr2=0x7f9190191130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.578+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 shutdown_connections 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:23.578+0000 7f9195aa8700 1 -- 172.21.15.3:0/2384696299 wait complete. 2024-05-08T05:42:24.514 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling the dashboard module... 2024-05-08T05:42:24.856 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:24 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2384696299' entity='client.admin' 2024-05-08T05:42:24.856 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:24 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:25 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3054608322' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T05:42:25.791 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:25 smithi003 ceph-mon[30137]: from='mgr.14118 172.21.15.3:0/3139377699' entity='mgr.x' 2024-05-08T05:42:25.792 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:25 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: ignoring --setuser ceph since I am not root 2024-05-08T05:42:25.792 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:25 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: ignoring --setgroup ceph since I am not root 2024-05-08T05:42:25.792 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:25 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:25.791+0000 7f0ae161e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:42:26.108 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:25.998+0000 7f0ae161e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:26.108 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:26.107+0000 7f0ae161e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.974+0000 7fbb57540700 1 Processor -- start 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.974+0000 7fbb57540700 1 -- start start 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.974+0000 7fbb57540700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb5010bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.974+0000 7fbb57540700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5007ce90 con 0x7fbb50109800 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.974+0000 7fbb552dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb5010bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.974+0000 7fbb552dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb5010bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59760/0 (socket says 172.21.15.3:59760) 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.975+0000 7fbb552dc700 1 -- 172.21.15.3:0/4114693047 learned_addr learned my addr 172.21.15.3:0/4114693047 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:26.236 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.975+0000 7fbb552dc700 1 -- 172.21.15.3:0/4114693047 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5010c130 con 0x7fbb50109800 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.975+0000 7fbb552dc700 1 --2- 172.21.15.3:0/4114693047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb5010bbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009e80 tx=0x7fbb4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d732ca053566ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.975+0000 7fbb47fff700 1 -- 172.21.15.3:0/4114693047 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb4c017070 con 0x7fbb50109800 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.975+0000 7fbb47fff700 1 -- 172.21.15.3:0/4114693047 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c00b8c0 con 0x7fbb50109800 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.975+0000 7fbb47fff700 1 -- 172.21.15.3:0/4114693047 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb4c004490 con 0x7fbb50109800 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 -- 172.21.15.3:0/4114693047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 msgr2=0x7fbb5010bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 --2- 172.21.15.3:0/4114693047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb5010bbf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009e80 tx=0x7fbb4c00b330 comp rx=0 tx=0).stop 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 -- 172.21.15.3:0/4114693047 shutdown_connections 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 --2- 172.21.15.3:0/4114693047 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb5010bbf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 -- 172.21.15.3:0/4114693047 >> 172.21.15.3:0/4114693047 conn(0x7fbb50103250 msgr2=0x7fbb50105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 -- 172.21.15.3:0/4114693047 shutdown_connections 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.976+0000 7fbb57540700 1 -- 172.21.15.3:0/4114693047 wait complete. 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb57540700 1 Processor -- start 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb57540700 1 -- start start 2024-05-08T05:42:26.237 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb57540700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb501a3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb57540700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5010d140 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb552dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb501a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb552dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb501a3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59766/0 (socket says 172.21.15.3:59766) 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb552dc700 1 -- 172.21.15.3:0/3054608322 learned_addr learned my addr 172.21.15.3:0/3054608322 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.977+0000 7fbb552dc700 1 -- 172.21.15.3:0/3054608322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb501a3880 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.978+0000 7fbb552dc700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb501a3340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b7b0 tx=0x7fbb4c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.978+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb4c020020 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.978+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb501a3b10 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.978+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c003c60 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.978+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb4c017040 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.978+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb501a6800 con 0x7fbb50109800 2024-05-08T05:42:26.238 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.979+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbb4c023020 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.979+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb50064b00 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.979+0000 7fbb467fc700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fbb3c03e470 0x7fbb3c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.979+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb4c00f050 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.979+0000 7fbb54adb700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fbb3c03e470 0x7fbb3c040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.980+0000 7fbb54adb700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fbb3c03e470 0x7fbb3c040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb40006fd0 tx=0x7fbb40006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:24.985+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb4c02a020 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.266+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fbb501a6dd0 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.580+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fbb4c02b260 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.582+0000 7fbb467fc700 1 -- 172.21.15.3:0/3054608322 <== mon.0 v2:172.21.15.3:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fbb4c02dc70 con 0x7fbb50109800 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fbb3c03e470 msgr2=0x7fbb3c040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fbb3c03e470 0x7fbb3c040930 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb40006fd0 tx=0x7fbb40006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 msgr2=0x7fbb501a3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:26.239 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb501a3340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00b7b0 tx=0x7fbb4c00bd50 comp rx=0 tx=0).stop 2024-05-08T05:42:26.240 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 shutdown_connections 2024-05-08T05:42:26.240 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7fbb3c03e470 0x7fbb3c040930 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.240 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 --2- 172.21.15.3:0/3054608322 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbb50109800 0x7fbb501a3340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:26.240 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 >> 172.21.15.3:0/3054608322 conn(0x7fbb50103250 msgr2=0x7fbb50103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:26.240 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 shutdown_connections 2024-05-08T05:42:26.240 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:25.584+0000 7fbb57540700 1 -- 172.21.15.3:0/3054608322 wait complete. 2024-05-08T05:42:26.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:26 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3054608322' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T05:42:26.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:26 smithi003 ceph-mon[30137]: mgrmap e9: x(active, since 19s) 2024-05-08T05:42:26.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:26.449+0000 7f0ae161e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:42:26.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:26.544+0000 7f0ae161e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:42:27.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:27 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2471692095' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:42:28.036 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:27 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:27.857+0000 7f0ae161e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.700+0000 7f7fcfa50700 1 Processor -- start 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.700+0000 7f7fcfa50700 1 -- start start 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.701+0000 7f7fcfa50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc810bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.701+0000 7f7fcfa50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc807d6a0 con 0x7f7fc8109890 2024-05-08T05:42:28.045 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.701+0000 7f7fcd7ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc810bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.701+0000 7f7fcd7ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc810bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59798/0 (socket says 172.21.15.3:59798) 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.701+0000 7f7fcd7ec700 1 -- 172.21.15.3:0/2402594221 learned_addr learned my addr 172.21.15.3:0/2402594221 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.702+0000 7f7fcd7ec700 1 -- 172.21.15.3:0/2402594221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc810c1c0 con 0x7f7fc8109890 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.702+0000 7f7fcd7ec700 1 --2- 172.21.15.3:0/2402594221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc810bc80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40098a0 tx=0x7f7fc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de720462253720a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.702+0000 7f7fbffff700 1 -- 172.21.15.3:0/2402594221 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc4004160 con 0x7f7fc8109890 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.702+0000 7f7fbffff700 1 -- 172.21.15.3:0/2402594221 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc4004300 con 0x7f7fc8109890 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.702+0000 7f7fbffff700 1 -- 172.21.15.3:0/2402594221 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc4011a20 con 0x7f7fc8109890 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2402594221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 msgr2=0x7f7fc810bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 --2- 172.21.15.3:0/2402594221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc810bc80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40098a0 tx=0x7f7fc4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2402594221 shutdown_connections 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 --2- 172.21.15.3:0/2402594221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc810bc80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2402594221 >> 172.21.15.3:0/2402594221 conn(0x7f7fc8103220 msgr2=0x7f7fc8105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:28.046 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2402594221 shutdown_connections 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.703+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2402594221 wait complete. 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcfa50700 1 Processor -- start 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcfa50700 1 -- start start 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcfa50700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcfa50700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc810d160 con 0x7f7fc8109890 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcd7ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcd7ec700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59808/0 (socket says 172.21.15.3:59808) 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcd7ec700 1 -- 172.21.15.3:0/2471692095 learned_addr learned my addr 172.21.15.3:0/2471692095 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcd7ec700 1 -- 172.21.15.3:0/2471692095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc819d220 con 0x7f7fc8109890 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fcd7ec700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc819cce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4004660 tx=0x7f7fc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.704+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc4003aa0 con 0x7f7fc8109890 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.705+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc819d4b0 con 0x7f7fc8109890 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.705+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fc401b8f0 con 0x7f7fc8109890 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.705+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fc4023900 con 0x7f7fc8109890 2024-05-08T05:42:28.047 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.705+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc81a0190 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.706+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc8064b00 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.706+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7fc4023bc0 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.706+0000 7f7fbe7fc700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 0x7f7fb4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.706+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7fc4053f90 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.706+0000 7f7fccfeb700 1 -- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 msgr2=0x7f7fb4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.706+0000 7f7fccfeb700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 0x7f7fb4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.711+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fc401f030 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.907+0000 7f7fccfeb700 1 -- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 msgr2=0x7f7fb4040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:26.907+0000 7f7fccfeb700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 0x7f7fb4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.036+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f7fc81a0400 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.036+0000 7f7fbe7fc700 1 -- 172.21.15.3:0/2471692095 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f7fc40538a0 con 0x7f7fc8109890 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 msgr2=0x7f7fb4040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 0x7f7fb4040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:28.048 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 msgr2=0x7f7fc819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc819cce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4004660 tx=0x7f7fc4011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 shutdown_connections 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f7fb403e4c0 0x7f7fb4040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 --2- 172.21.15.3:0/2471692095 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fc8109890 0x7f7fc819cce0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 >> 172.21.15.3:0/2471692095 conn(0x7f7fc8103220 msgr2=0x7f7fc8103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 shutdown_connections 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:27.039+0000 7f7fcfa50700 1 -- 172.21.15.3:0/2471692095 wait complete. 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for the mgr to restart... 2024-05-08T05:42:28.049 INFO:teuthology.orchestra.run.smithi003.stdout:Waiting for mgr epoch 9... 2024-05-08T05:42:28.339 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:28 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:28.036+0000 7f0ae161e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:28.339 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:28 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:28.218+0000 7f0ae161e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:42:28.630 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:28 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:28.531+0000 7f0ae161e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:42:28.630 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:28 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:28.629+0000 7f0ae161e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:42:28.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:28 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:28.843+0000 7f0ae161e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:42:29.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:29 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:29.046+0000 7f0ae161e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:42:30.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:29 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:29.683+0000 7f0ae161e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:42:30.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:29 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:29.915+0000 7f0ae161e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:42:30.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:30 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:30.362+0000 7f0ae161e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:42:30.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:30 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:30.463+0000 7f0ae161e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:42:31.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:30 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:30.849+0000 7f0ae161e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:42:31.581 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:31.225+0000 7f0ae161e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:42:31.581 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:31.323+0000 7f0ae161e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:42:31.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:31.580+0000 7f0ae161e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:42:31.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:31.721+0000 7f0ae161e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:42:31.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:31.821+0000 7f0ae161e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:42:32.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:32 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:32.162+0000 7f0ae161e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:42:32.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:32 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:32.259+0000 7f0ae161e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:42:33.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:32 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:32.892+0000 7f0ae161e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:42:33.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:42:33 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:42:33.119+0000 7f0ae161e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: Active manager daemon x restarted 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: Activating manager daemon x 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: mgrmap e10: x(active, starting, since 0.00397134s) 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:42:33.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:42:33.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:42:33.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: Manager daemon x is now available 2024-05-08T05:42:33.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:34.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:42:34.383 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:42:34.384 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:34 smithi003 ceph-mon[30137]: mgrmap e11: x(active, since 1.00606s) 2024-05-08T05:42:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:35 smithi003 ceph-mon[30137]: [08/May/2024:05:42:33] ENGINE Bus STARTING 2024-05-08T05:42:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:35 smithi003 ceph-mon[30137]: [08/May/2024:05:42:33] ENGINE Serving on http://172.21.15.3:8765 2024-05-08T05:42:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:35 smithi003 ceph-mon[30137]: [08/May/2024:05:42:33] ENGINE Serving on https://172.21.15.3:7150 2024-05-08T05:42:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:35 smithi003 ceph-mon[30137]: [08/May/2024:05:42:33] ENGINE Bus STARTED 2024-05-08T05:42:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:35 smithi003 ceph-mon[30137]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:42:35.291 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:35 smithi003 ceph-mon[30137]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.445+0000 7f2bf6268700 1 Processor -- start 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.445+0000 7f2bf6268700 1 -- start start 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.445+0000 7f2bf6268700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf01080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.445+0000 7f2bf6268700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0108680 con 0x7f2bf0107c90 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf01080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:35.304 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf01080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59810/0 (socket says 172.21.15.3:59810) 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beffff700 1 -- 172.21.15.3:0/3879711312 learned_addr learned my addr 172.21.15.3:0/3879711312 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beffff700 1 -- 172.21.15.3:0/3879711312 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf0108e90 con 0x7f2bf0107c90 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beffff700 1 --2- 172.21.15.3:0/3879711312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf01080b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2be000bb40 tx=0x7f2be000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8949d0127509595 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beeffd700 1 -- 172.21.15.3:0/3879711312 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2be000dbe0 con 0x7f2bf0107c90 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beeffd700 1 -- 172.21.15.3:0/3879711312 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00044e0 con 0x7f2bf0107c90 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.446+0000 7f2beeffd700 1 -- 172.21.15.3:0/3879711312 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2be0010470 con 0x7f2bf0107c90 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 -- 172.21.15.3:0/3879711312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 msgr2=0x7f2bf01080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3879711312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf01080b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2be000bb40 tx=0x7f2be000be50 comp rx=0 tx=0).stop 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 -- 172.21.15.3:0/3879711312 shutdown_connections 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3879711312 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf01080b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.305 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 -- 172.21.15.3:0/3879711312 >> 172.21.15.3:0/3879711312 conn(0x7f2bf0103210 msgr2=0x7f2bf0105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 -- 172.21.15.3:0/3879711312 shutdown_connections 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.447+0000 7f2bf6268700 1 -- 172.21.15.3:0/3879711312 wait complete. 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2bf6268700 1 Processor -- start 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2bf6268700 1 -- start start 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2bf6268700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2bf6268700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf007c840 con 0x7f2bf0107c90 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2beffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2beffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59826/0 (socket says 172.21.15.3:59826) 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.448+0000 7f2beffff700 1 -- 172.21.15.3:0/3421790246 learned_addr learned my addr 172.21.15.3:0/3421790246 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2beffff700 1 -- 172.21.15.3:0/3421790246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf019f450 con 0x7f2bf0107c90 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2beffff700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf019ef10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2be000d650 tx=0x7f2be0003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:35.306 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2be0010770 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf019f6e0 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be001d070 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2be0021bf0 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.449+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a23d0 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.450+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f2be001f030 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.450+0000 7f2bed7fa700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.450+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 --> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2bf006a5b0 con 0x7f2bdc03e470 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.450+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2be00539c0 con 0x7f2bf0107c90 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.450+0000 7f2bef7fe700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 msgr2=0x7f2bdc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.450+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.651+0000 7f2bef7fe700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 msgr2=0x7f2bdc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:28.651+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:42:35.307 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.052+0000 7f2bef7fe700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 msgr2=0x7f2bdc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.052+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.853+0000 7f2bef7fe700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 msgr2=0x7f2bdc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:29.853+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:31.454+0000 7f2bef7fe700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 msgr2=0x7f2bdc040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/174548513 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:31.454+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.123+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f2be0021410 con 0x7f2bf0107c90 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.124+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 msgr2=0x7f2bdc040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:33.124+0000 7f2bed7fa700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.125+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f2be0019cd0 con 0x7f2bf0107c90 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.125+0000 7f2bed7fa700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2bdc041d00 0x7f2bdc0440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.125+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2bf006a5b0 con 0x7f2bdc041d00 2024-05-08T05:42:35.308 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.125+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2bdc041d00 0x7f2bdc0440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.126+0000 7f2bef7fe700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2bdc041d00 0x7f2bdc0440f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8003a10 tx=0x7f2bd80092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.127+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f2bf006a5b0 con 0x7f2bdc041d00 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2bf00638e0 con 0x7f2bdc041d00 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bed7fa700 1 -- 172.21.15.3:0/3421790246 <== mgr.14150 v2:172.21.15.3:6800/2793699832 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f2bf00638e0 con 0x7f2bdc041d00 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2bdc041d00 msgr2=0x7f2bdc0440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2bdc041d00 0x7f2bdc0440f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8003a10 tx=0x7f2bd80092b0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 msgr2=0x7f2bf019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf019ef10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2be000d650 tx=0x7f2be0003ff0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 shutdown_connections 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2bdc041d00 0x7f2bdc0440f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2bf0107c90 0x7f2bf019ef10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 --2- 172.21.15.3:0/3421790246 >> [v2:172.21.15.3:6800/174548513,v1:172.21.15.3:6801/174548513] conn(0x7f2bdc03e470 0x7f2bdc040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:35.309 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 >> 172.21.15.3:0/3421790246 conn(0x7f2bf0103210 msgr2=0x7f2bf0191660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:35.310 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 shutdown_connections 2024-05-08T05:42:35.310 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:34.136+0000 7f2bf6268700 1 -- 172.21.15.3:0/3421790246 wait complete. 2024-05-08T05:42:35.310 INFO:teuthology.orchestra.run.smithi003.stdout:mgr epoch 9 is available 2024-05-08T05:42:35.310 INFO:teuthology.orchestra.run.smithi003.stdout:Generating a dashboard self-signed certificate... 2024-05-08T05:42:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:36 smithi003 ceph-mon[30137]: mgrmap e12: x(active, since 2s) 2024-05-08T05:42:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.773+0000 7fcb1a36e700 1 Processor -- start 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.773+0000 7fcb1a36e700 1 -- start start 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.773+0000 7fcb1a36e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb141080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.773+0000 7fcb1a36e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14108680 con 0x7fcb14107c90 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.773+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb141080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:36.814 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb141080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36664/0 (socket says 172.21.15.3:36664) 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb13fff700 1 -- 172.21.15.3:0/1924656288 learned_addr learned my addr 172.21.15.3:0/1924656288 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb13fff700 1 -- 172.21.15.3:0/1924656288 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb14108e90 con 0x7fcb14107c90 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb13fff700 1 --2- 172.21.15.3:0/1924656288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb141080b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400bb40 tx=0x7fcb0400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8c21382b863dd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb12ffd700 1 -- 172.21.15.3:0/1924656288 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb0400dbe0 con 0x7fcb14107c90 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb12ffd700 1 -- 172.21.15.3:0/1924656288 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb040044e0 con 0x7fcb14107c90 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.774+0000 7fcb12ffd700 1 -- 172.21.15.3:0/1924656288 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04010470 con 0x7fcb14107c90 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 -- 172.21.15.3:0/1924656288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 msgr2=0x7fcb141080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 --2- 172.21.15.3:0/1924656288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb141080b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400bb40 tx=0x7fcb0400be50 comp rx=0 tx=0).stop 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 -- 172.21.15.3:0/1924656288 shutdown_connections 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 --2- 172.21.15.3:0/1924656288 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb141080b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 -- 172.21.15.3:0/1924656288 >> 172.21.15.3:0/1924656288 conn(0x7fcb14103210 msgr2=0x7fcb14105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 -- 172.21.15.3:0/1924656288 shutdown_connections 2024-05-08T05:42:36.815 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 -- 172.21.15.3:0/1924656288 wait complete. 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.775+0000 7fcb1a36e700 1 Processor -- start 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb1a36e700 1 -- start start 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb1a36e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb1419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb1a36e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1407c840 con 0x7fcb14107c90 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb1419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb13fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb1419eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36668/0 (socket says 172.21.15.3:36668) 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb13fff700 1 -- 172.21.15.3:0/988241630 learned_addr learned my addr 172.21.15.3:0/988241630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb13fff700 1 -- 172.21.15.3:0/988241630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1419f420 con 0x7fcb14107c90 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.776+0000 7fcb13fff700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb1419eee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400d650 tx=0x7fcb04003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.777+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04010770 con 0x7fcb14107c90 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.777+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1419f6b0 con 0x7fcb14107c90 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.777+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb0401d070 con 0x7fcb14107c90 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.777+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb141a23a0 con 0x7fcb14107c90 2024-05-08T05:42:36.816 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.777+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb04021bf0 con 0x7fcb14107c90 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.778+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fcb0401f030 con 0x7fcb14107c90 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.778+0000 7fcb117fa700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcafc03e470 0x7fcafc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.778+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcb04053aa0 con 0x7fcb14107c90 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.778+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb14064b00 con 0x7fcb14107c90 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.778+0000 7fcb137fe700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcafc03e470 0x7fcafc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.779+0000 7fcb137fe700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcafc03e470 0x7fcafc040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb08006fd0 tx=0x7fcb08006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:35.784+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcb04015070 con 0x7fcb14107c90 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.042+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fcb1402e7b0 con 0x7fcafc03e470 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.163+0000 7fcb117fa700 1 -- 172.21.15.3:0/988241630 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fcb1402e7b0 con 0x7fcafc03e470 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcafc03e470 msgr2=0x7fcafc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcafc03e470 0x7fcafc040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcb08006fd0 tx=0x7fcb08006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 msgr2=0x7fcb1419eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb1419eee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcb0400d650 tx=0x7fcb04003ff0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.817 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 shutdown_connections 2024-05-08T05:42:36.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcafc03e470 0x7fcafc040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 --2- 172.21.15.3:0/988241630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fcb14107c90 0x7fcb1419eee0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:36.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 >> 172.21.15.3:0/988241630 conn(0x7fcb14103210 msgr2=0x7fcb14191660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:36.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 shutdown_connections 2024-05-08T05:42:36.818 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:36.167+0000 7fcb1a36e700 1 -- 172.21.15.3:0/988241630 wait complete. 2024-05-08T05:42:36.818 INFO:teuthology.orchestra.run.smithi003.stdout:Creating initial admin user... 2024-05-08T05:42:37.605 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:37 smithi003 ceph-mon[30137]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:38.361 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$DQYownFkbcYx34MSMsIJkejT.1Q1ZsoEt6uAOE/t0.f5zGvdillQO", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715146957, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.356+0000 7f9a5b371700 1 Processor -- start 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5b371700 1 -- start start 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5b371700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a54109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5b371700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9a541097d0 con 0x7f9a54108de0 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5910d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a54109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5910d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a54109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36680/0 (socket says 172.21.15.3:36680) 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5910d700 1 -- 172.21.15.3:0/3975823062 learned_addr learned my addr 172.21.15.3:0/3975823062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.357+0000 7f9a5910d700 1 -- 172.21.15.3:0/3975823062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a54109fe0 con 0x7f9a54108de0 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a5910d700 1 --2- 172.21.15.3:0/3975823062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a54109200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a500098a0 tx=0x7f9a50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6d9234738083e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a47fff700 1 -- 172.21.15.3:0/3975823062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a50004160 con 0x7f9a54108de0 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a47fff700 1 -- 172.21.15.3:0/3975823062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a50004300 con 0x7f9a54108de0 2024-05-08T05:42:38.362 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a47fff700 1 -- 172.21.15.3:0/3975823062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a50011a20 con 0x7f9a54108de0 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a5b371700 1 -- 172.21.15.3:0/3975823062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 msgr2=0x7f9a54109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a5b371700 1 --2- 172.21.15.3:0/3975823062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a54109200 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a500098a0 tx=0x7f9a50009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.358+0000 7f9a5b371700 1 -- 172.21.15.3:0/3975823062 shutdown_connections 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.359+0000 7f9a5b371700 1 --2- 172.21.15.3:0/3975823062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a54109200 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.359+0000 7f9a5b371700 1 -- 172.21.15.3:0/3975823062 >> 172.21.15.3:0/3975823062 conn(0x7f9a541043a0 msgr2=0x7f9a541067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.359+0000 7f9a5b371700 1 -- 172.21.15.3:0/3975823062 shutdown_connections 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.359+0000 7f9a5b371700 1 -- 172.21.15.3:0/3975823062 wait complete. 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.359+0000 7f9a5b371700 1 Processor -- start 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.359+0000 7f9a5b371700 1 -- start start 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5b371700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a5419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5b371700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5410d390 con 0x7f9a54108de0 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5910d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a5419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:38.363 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5910d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a5419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36682/0 (socket says 172.21.15.3:36682) 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5910d700 1 -- 172.21.15.3:0/470989622 learned_addr learned my addr 172.21.15.3:0/470989622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5910d700 1 -- 172.21.15.3:0/470989622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5419d1b0 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5910d700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a5419cc70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009520 tx=0x7f9a50011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a500039e0 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5419d440 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a5001b8f0 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a50019770 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.360+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a541a0130 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.362+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f9a50019a70 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.362+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a54064b00 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.362+0000 7f9a467fc700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9a4003e4c0 0x7f9a40040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.362+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a50053fd0 con 0x7f9a54108de0 2024-05-08T05:42:38.364 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.362+0000 7f9a5890c700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9a4003e4c0 0x7f9a40040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.363+0000 7f9a5890c700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9a4003e4c0 0x7f9a40040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a48006fd0 tx=0x7f9a48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.367+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9a5001f030 con 0x7f9a54108de0 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.604+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f9a541a0300 con 0x7f9a4003e4c0 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.851+0000 7f9a467fc700 1 -- 172.21.15.3:0/470989622 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f9a541a0300 con 0x7f9a4003e4c0 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.853+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9a4003e4c0 msgr2=0x7f9a40040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.853+0000 7f9a5b371700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9a4003e4c0 0x7f9a40040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a48006fd0 tx=0x7f9a48006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.853+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 msgr2=0x7f9a5419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.853+0000 7f9a5b371700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a5419cc70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009520 tx=0x7f9a50011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.854+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 shutdown_connections 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.854+0000 7f9a5b371700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9a4003e4c0 0x7f9a40040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.854+0000 7f9a5b371700 1 --2- 172.21.15.3:0/470989622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9a54108de0 0x7f9a5419cc70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.854+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 >> 172.21.15.3:0/470989622 conn(0x7f9a541043a0 msgr2=0x7f9a54104d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.854+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 shutdown_connections 2024-05-08T05:42:38.365 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:37.854+0000 7f9a5b371700 1 -- 172.21.15.3:0/470989622 wait complete. 2024-05-08T05:42:38.366 INFO:teuthology.orchestra.run.smithi003.stdout:Fetching dashboard port number... 2024-05-08T05:42:38.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:38.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:38 smithi003 ceph-mon[30137]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:38.707 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:39.638 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:39 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/99728693' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T05:42:39.645 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.814+0000 7fc77707d700 1 Processor -- start 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.814+0000 7fc77707d700 1 -- start start 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc77707d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc7701091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc77707d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc7701097b0 con 0x7fc770108dc0 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc774e19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc7701091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc774e19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc7701091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36686/0 (socket says 172.21.15.3:36686) 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc774e19700 1 -- 172.21.15.3:0/121565191 learned_addr learned my addr 172.21.15.3:0/121565191 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc774e19700 1 -- 172.21.15.3:0/121565191 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc770109fc0 con 0x7fc770108dc0 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc774e19700 1 --2- 172.21.15.3:0/121565191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc7701091e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7640098a0 tx=0x7fc764009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce202cae0c2d5838 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.815+0000 7fc76f7fe700 1 -- 172.21.15.3:0/121565191 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc764004160 con 0x7fc770108dc0 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc76f7fe700 1 -- 172.21.15.3:0/121565191 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764004300 con 0x7fc770108dc0 2024-05-08T05:42:39.646 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc76f7fe700 1 -- 172.21.15.3:0/121565191 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc764011a20 con 0x7fc770108dc0 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 -- 172.21.15.3:0/121565191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 msgr2=0x7fc7701091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 --2- 172.21.15.3:0/121565191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc7701091e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7640098a0 tx=0x7fc764009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 -- 172.21.15.3:0/121565191 shutdown_connections 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 --2- 172.21.15.3:0/121565191 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc7701091e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 -- 172.21.15.3:0/121565191 >> 172.21.15.3:0/121565191 conn(0x7fc7701043a0 msgr2=0x7fc7701067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 -- 172.21.15.3:0/121565191 shutdown_connections 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.816+0000 7fc77707d700 1 -- 172.21.15.3:0/121565191 wait complete. 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.817+0000 7fc77707d700 1 Processor -- start 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.817+0000 7fc77707d700 1 -- start start 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.817+0000 7fc77707d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc77019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc77707d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc77010d370 con 0x7fc770108dc0 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc774e19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc77019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc774e19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc77019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36704/0 (socket says 172.21.15.3:36704) 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc774e19700 1 -- 172.21.15.3:0/99728693 learned_addr learned my addr 172.21.15.3:0/99728693 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc774e19700 1 -- 172.21.15.3:0/99728693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77019d2d0 con 0x7fc770108dc0 2024-05-08T05:42:39.647 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc774e19700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc77019cd90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc764000c00 tx=0x7fc764011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc764003aa0 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76401b8f0 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77019d560 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.818+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc764023890 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.819+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7701a0210 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.820+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fc76402c3f0 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.820+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc770064b00 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.820+0000 7fc76dffb700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc76003e4c0 0x7fc760040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.820+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc764053f50 con 0x7fc770108dc0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.820+0000 7fc76ffff700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc76003e4c0 0x7fc760040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.820+0000 7fc76ffff700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc76003e4c0 0x7fc760040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006fd0 tx=0x7fc75c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:39.648 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:38.825+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc76401f030 con 0x7fc770108dc0 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.050+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc7701a0890 con 0x7fc770108dc0 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.051+0000 7fc76dffb700 1 -- 172.21.15.3:0/99728693 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc76400f2e0 con 0x7fc770108dc0 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc76003e4c0 msgr2=0x7fc760040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc76003e4c0 0x7fc760040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc75c006fd0 tx=0x7fc75c006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 msgr2=0x7fc77019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc77019cd90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc764000c00 tx=0x7fc764011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 shutdown_connections 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc76003e4c0 0x7fc760040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 --2- 172.21.15.3:0/99728693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc770108dc0 0x7fc77019cd90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 >> 172.21.15.3:0/99728693 conn(0x7fc7701043a0 msgr2=0x7fc770104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:39.649 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 shutdown_connections 2024-05-08T05:42:39.650 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:39.053+0000 7fc77707d700 1 -- 172.21.15.3:0/99728693 wait complete. 2024-05-08T05:42:39.653 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:42:39.653 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout disabled 2024-05-08T05:42:39.661 INFO:teuthology.orchestra.run.smithi003.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:42:39.661 INFO:teuthology.orchestra.run.smithi003.stdout:systemctl: stdout inactive 2024-05-08T05:42:39.661 INFO:teuthology.orchestra.run.smithi003.stdout:firewalld.service is not enabled 2024-05-08T05:42:39.661 INFO:teuthology.orchestra.run.smithi003.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T05:42:39.662 INFO:teuthology.orchestra.run.smithi003.stdout:Ceph Dashboard is now available at: 2024-05-08T05:42:39.662 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:39.663 INFO:teuthology.orchestra.run.smithi003.stdout: URL: https://smithi003.front.sepia.ceph.com:8443/ 2024-05-08T05:42:39.663 INFO:teuthology.orchestra.run.smithi003.stdout: User: admin 2024-05-08T05:42:39.663 INFO:teuthology.orchestra.run.smithi003.stdout: Password: 2jj5gj3f8s 2024-05-08T05:42:39.663 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:39.663 INFO:teuthology.orchestra.run.smithi003.stdout:Saving cluster configuration to /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config directory 2024-05-08T05:42:39.663 INFO:teuthology.orchestra.run.smithi003.stdout:Enabling autotune for osd_memory_target 2024-05-08T05:42:40.643 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:40 smithi003 ceph-mon[30137]: mgrmap e13: x(active, since 6s) 2024-05-08T05:42:41.631 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811b3c1700 1 Processor -- start 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811b3c1700 1 -- start start 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811b3c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f8114109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811b3c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81141097d0 con 0x7f8114108de0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811915d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f8114109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811915d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f8114109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36708/0 (socket says 172.21.15.3:36708) 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.860+0000 7f811915d700 1 -- 172.21.15.3:0/783694767 learned_addr learned my addr 172.21.15.3:0/783694767 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f811915d700 1 -- 172.21.15.3:0/783694767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8114109fe0 con 0x7f8114108de0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f811915d700 1 --2- 172.21.15.3:0/783694767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f8114109200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81100098a0 tx=0x7f8110009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30e1f5e7c2531ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f810bfff700 1 -- 172.21.15.3:0/783694767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110004160 con 0x7f8114108de0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f810bfff700 1 -- 172.21.15.3:0/783694767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8110004300 con 0x7f8114108de0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f810bfff700 1 -- 172.21.15.3:0/783694767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110011a20 con 0x7f8114108de0 2024-05-08T05:42:41.632 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f811b3c1700 1 -- 172.21.15.3:0/783694767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 msgr2=0x7f8114109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.861+0000 7f811b3c1700 1 --2- 172.21.15.3:0/783694767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f8114109200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81100098a0 tx=0x7f8110009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 -- 172.21.15.3:0/783694767 shutdown_connections 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 --2- 172.21.15.3:0/783694767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f8114109200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 -- 172.21.15.3:0/783694767 >> 172.21.15.3:0/783694767 conn(0x7f81141043a0 msgr2=0x7f81141067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 -- 172.21.15.3:0/783694767 shutdown_connections 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 -- 172.21.15.3:0/783694767 wait complete. 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 Processor -- start 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.862+0000 7f811b3c1700 1 -- start start 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811b3c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f811419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811b3c1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f811410d390 con 0x7f8114108de0 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811915d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f811419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811915d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f811419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36720/0 (socket says 172.21.15.3:36720) 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811915d700 1 -- 172.21.15.3:0/212015282 learned_addr learned my addr 172.21.15.3:0/212015282 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811915d700 1 -- 172.21.15.3:0/212015282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811419d210 con 0x7f8114108de0 2024-05-08T05:42:41.633 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811915d700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f811419ccd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8110009520 tx=0x7f8110011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81100039e0 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f811001b8f0 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811419d4a0 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8110019770 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.863+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a0180 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.864+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8114064b00 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.864+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f811001b440 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.865+0000 7f810a7fc700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f810003e560 0x7f8100040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.865+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8110054060 con 0x7f8114108de0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.865+0000 7f811895c700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f810003e560 0x7f8100040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.865+0000 7f811895c700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f810003e560 0x7f8100040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8104006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:41.634 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:40.870+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f811001f030 con 0x7f8114108de0 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.083+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f81141a07e0 con 0x7f8114108de0 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.083+0000 7f810a7fc700 1 -- 172.21.15.3:0/212015282 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f8110056160 con 0x7f8114108de0 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f810003e560 msgr2=0x7f8100040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f810003e560 0x7f8100040a20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8104006fd0 tx=0x7f8104006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 msgr2=0x7f811419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f811419ccd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8110009520 tx=0x7f8110011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 shutdown_connections 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f810003e560 0x7f8100040a20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 --2- 172.21.15.3:0/212015282 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8114108de0 0x7f811419ccd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 >> 172.21.15.3:0/212015282 conn(0x7f81141043a0 msgr2=0x7f8114104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 shutdown_connections 2024-05-08T05:42:41.635 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:41.086+0000 7f811b3c1700 1 -- 172.21.15.3:0/212015282 wait complete. 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.067+0000 7f2fdce73700 1 Processor -- start 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.067+0000 7f2fdce73700 1 -- start start 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fdce73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd8109200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fdce73700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd81097d0 con 0x7f2fd8108de0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fd659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd8109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fd659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd8109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36722/0 (socket says 172.21.15.3:36722) 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fd659c700 1 -- 172.21.15.3:0/729711241 learned_addr learned my addr 172.21.15.3:0/729711241 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fd659c700 1 -- 172.21.15.3:0/729711241 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd8109fe0 con 0x7f2fd8108de0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.068+0000 7f2fd659c700 1 --2- 172.21.15.3:0/729711241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd8109200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80098a0 tx=0x7f2fc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1857c9f7f5f5e12 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fd559a700 1 -- 172.21.15.3:0/729711241 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fc8004160 con 0x7f2fd8108de0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fd559a700 1 -- 172.21.15.3:0/729711241 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc8004300 con 0x7f2fd8108de0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fd559a700 1 -- 172.21.15.3:0/729711241 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fc8011a20 con 0x7f2fd8108de0 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 -- 172.21.15.3:0/729711241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 msgr2=0x7f2fd8109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:43.013 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 --2- 172.21.15.3:0/729711241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd8109200 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80098a0 tx=0x7f2fc8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 -- 172.21.15.3:0/729711241 shutdown_connections 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 --2- 172.21.15.3:0/729711241 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd8109200 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 -- 172.21.15.3:0/729711241 >> 172.21.15.3:0/729711241 conn(0x7f2fd81043a0 msgr2=0x7f2fd81067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 -- 172.21.15.3:0/729711241 shutdown_connections 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.069+0000 7f2fdce73700 1 -- 172.21.15.3:0/729711241 wait complete. 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.070+0000 7f2fdce73700 1 Processor -- start 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.070+0000 7f2fdce73700 1 -- start start 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.070+0000 7f2fdce73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.070+0000 7f2fdce73700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd810d390 con 0x7f2fd8108de0 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fd659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fd659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36734/0 (socket says 172.21.15.3:36734) 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fd659c700 1 -- 172.21.15.3:0/1785655432 learned_addr learned my addr 172.21.15.3:0/1785655432 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fd659c700 1 -- 172.21.15.3:0/1785655432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd819d1f0 con 0x7f2fd8108de0 2024-05-08T05:42:43.014 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fd659c700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd819ccb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80116e0 tx=0x7f2fc8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fc80039e0 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fc801b8f0 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd819d480 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fc8023760 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.071+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd81a0170 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.072+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f2fc801b440 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.072+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd8064b00 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.073+0000 7f2fcf7fe700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2fc403e570 0x7f2fc4040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.073+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2fc8054000 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.073+0000 7f2fd5d9b700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2fc403e570 0x7f2fc4040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.073+0000 7f2fd5d9b700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2fc403e570 0x7f2fc4040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0006fd0 tx=0x7f2fc0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.078+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2fc801f030 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.409+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2fd81a03e0 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.412+0000 7f2fcf7fe700 1 -- 172.21.15.3:0/1785655432 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f2fc8018590 con 0x7f2fd8108de0 2024-05-08T05:42:43.015 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2fc403e570 msgr2=0x7f2fc4040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2fc403e570 0x7f2fc4040a30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2fc0006fd0 tx=0x7f2fc0006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 msgr2=0x7f2fd819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd819ccb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80116e0 tx=0x7f2fc8011aa0 comp rx=0 tx=0).stop 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 shutdown_connections 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2fc403e570 0x7f2fc4040a30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 --2- 172.21.15.3:0/1785655432 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2fd8108de0 0x7f2fd819ccb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.414+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 >> 172.21.15.3:0/1785655432 conn(0x7f2fd81043a0 msgr2=0x7f2fd8104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.415+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 shutdown_connections 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr 2024-05-08T05:42:42.415+0000 7f2fdce73700 1 -- 172.21.15.3:0/1785655432 wait complete. 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T05:42:43.016 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: ceph telemetry on 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout:For more information see: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:43.017 INFO:teuthology.orchestra.run.smithi003.stdout:Bootstrap complete. 2024-05-08T05:42:43.058 INFO:tasks.cephadm:Fetching config... 2024-05-08T05:42:43.058 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:42:43.058 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T05:42:43.076 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T05:42:43.076 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:42:43.076 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T05:42:43.133 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T05:42:43.133 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:42:43.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T05:42:43.200 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T05:42:43.200 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:42:43.200 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T05:42:43.257 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T05:42:43.257 DEBUG:teuthology.orchestra.run.smithi003:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:42:43.335 INFO:teuthology.orchestra.run.smithi003.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:43.347 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:42:43.386 INFO:teuthology.orchestra.run.smithi089.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:43.397 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:42:43.437 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjlkduNjLx22rwGflTOcZORw4itRYm+4RewuJXCPsHeHUvlwyucLDir1KA+qY/9nNh/hwUVm1hXbqNmyxihVhz/wkom/yFy0qDIrOZ5yc6laNulw4wWnRKOfW3TL+GDhNy8tfZIDpTWYOAABIzMnGc3EO+haGm6QSjg6qF3QYLj8MId4Gmz2MIdeX4Yj04aTJdV2bWbHZTmCgp12chJow5kxc61SWICUItPyCxRrZW69lqr6KJV3k8AluUdRBoPF5iXI67HsrMMOB6XxLbCWzEI+ihOAu2YRPMDyTqj7rvGCrnbNeB2TEYjEVFCFeGBAYbdee4JlYK+S0H9ZbYQsOcMklB8t1CXZkXah0eiVg8fzOYCahoKFKsbZwa9DjECNzX7b8DenM/Ldsn1ypiks518W3hrSgxwPQ0qWauv2tY6CJfF7Rc9RoyFrqUi1VLXJGF7mtWfQ+RZEQBijMmNZNUsA8DzMgqOfT9qmlE9UPAkqhAJA91cpWC3QrpdAt+nMM= ceph-708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:42:43.449 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T05:42:43.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:43 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1785655432' entity='client.admin' 2024-05-08T05:42:43.696 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:42:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 -- 172.21.15.3:0/1305945339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 msgr2=0x7f5870105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 --2- 172.21.15.3:0/1305945339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870105d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5860009910 tx=0x7f5860009c20 comp rx=0 tx=0).stop 2024-05-08T05:42:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 -- 172.21.15.3:0/1305945339 shutdown_connections 2024-05-08T05:42:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 --2- 172.21.15.3:0/1305945339 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870105d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 -- 172.21.15.3:0/1305945339 >> 172.21.15.3:0/1305945339 conn(0x7f58700fd3b0 msgr2=0x7f58700ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 -- 172.21.15.3:0/1305945339 shutdown_connections 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.301+0000 7f587579c700 1 -- 172.21.15.3:0/1305945339 wait complete. 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.302+0000 7f587579c700 1 Processor -- start 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.302+0000 7f587579c700 1 -- start start 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.302+0000 7f587579c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870103220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.302+0000 7f587579c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5870108df0 con 0x7f58701039a0 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.302+0000 7f586effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870103220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f586effd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870103220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52502/0 (socket says 172.21.15.3:52502) 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f586effd700 1 -- 172.21.15.3:0/4055719515 learned_addr learned my addr 172.21.15.3:0/4055719515 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f586effd700 1 -- 172.21.15.3:0/4055719515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5870103760 con 0x7f58701039a0 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f586effd700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870103220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5860005190 tx=0x7f5860004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5860027070 con 0x7f58701039a0 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5860022470 con 0x7f58701039a0 2024-05-08T05:42:46.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5870101950 con 0x7f58701039a0 2024-05-08T05:42:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f586000f5f0 con 0x7f58701039a0 2024-05-08T05:42:46.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.303+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5870101e90 con 0x7f58701039a0 2024-05-08T05:42:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.304+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5870109570 con 0x7f58701039a0 2024-05-08T05:42:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.304+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f586002a070 con 0x7f58701039a0 2024-05-08T05:42:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.305+0000 7f5857fff700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f585803e570 0x7f5858040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.305+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f586001e030 con 0x7f58701039a0 2024-05-08T05:42:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.305+0000 7f586e7fc700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f585803e570 0x7f5858040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:46.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.305+0000 7f586e7fc700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f585803e570 0x7f5858040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:46.311 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.311+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5860013070 con 0x7f58701039a0 2024-05-08T05:42:46.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.529+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5870191890 con 0x7f58701039a0 2024-05-08T05:42:46.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.532+0000 7f5857fff700 1 -- 172.21.15.3:0/4055719515 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5860031070 con 0x7f58701039a0 2024-05-08T05:42:46.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f585803e570 msgr2=0x7f5858040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f585803e570 0x7f5858040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5864006fd0 tx=0x7f5864006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 msgr2=0x7f5870103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870103220 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5860005190 tx=0x7f5860004f80 comp rx=0 tx=0).stop 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 shutdown_connections 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f585803e570 0x7f5858040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 --2- 172.21.15.3:0/4055719515 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58701039a0 0x7f5870103220 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 >> 172.21.15.3:0/4055719515 conn(0x7f58700fd3b0 msgr2=0x7f58700fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 shutdown_connections 2024-05-08T05:42:46.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:46.535+0000 7f587579c700 1 -- 172.21.15.3:0/4055719515 wait complete. 2024-05-08T05:42:47.603 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T05:42:47.604 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T05:42:47.848 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/4055719515' entity='client.admin' 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi003", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 -- 172.21.15.3:0/2383222333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 msgr2=0x7fa6b0105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 --2- 172.21.15.3:0/2383222333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0105e40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009910 tx=0x7fa6ac009c20 comp rx=0 tx=0).stop 2024-05-08T05:42:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 -- 172.21.15.3:0/2383222333 shutdown_connections 2024-05-08T05:42:49.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 --2- 172.21.15.3:0/2383222333 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0105e40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 -- 172.21.15.3:0/2383222333 >> 172.21.15.3:0/2383222333 conn(0x7fa6b00fd3a0 msgr2=0x7fa6b00ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 -- 172.21.15.3:0/2383222333 shutdown_connections 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.437+0000 7fa6b7a92700 1 -- 172.21.15.3:0/2383222333 wait complete. 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b7a92700 1 Processor -- start 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b7a92700 1 -- start start 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b7a92700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0103280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b7a92700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0108e40 con 0x7fa6b0103a50 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b582e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0103280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b582e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0103280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52520/0 (socket says 172.21.15.3:52520) 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b582e700 1 -- 172.21.15.3:0/499956855 learned_addr learned my addr 172.21.15.3:0/499956855 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b582e700 1 -- 172.21.15.3:0/499956855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b01037c0 con 0x7fa6b0103a50 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.438+0000 7fa6b582e700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0103280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac005b40 tx=0x7fa6ac004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.439+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac027070 con 0x7fa6b0103a50 2024-05-08T05:42:49.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.439+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ac00be10 con 0x7fa6b0103a50 2024-05-08T05:42:49.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.439+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b0101a00 con 0x7fa6b0103a50 2024-05-08T05:42:49.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.439+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac022de0 con 0x7fa6b0103a50 2024-05-08T05:42:49.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.439+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b0101ee0 con 0x7fa6b0103a50 2024-05-08T05:42:49.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.440+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fa6ac02a070 con 0x7fa6b0103a50 2024-05-08T05:42:49.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.440+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b01095c0 con 0x7fa6b0103a50 2024-05-08T05:42:49.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.440+0000 7fa6a6ffd700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa69c03e570 0x7fa69c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:49.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.440+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa6ac01e030 con 0x7fa6b0103a50 2024-05-08T05:42:49.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.440+0000 7fa6b502d700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa69c03e570 0x7fa69c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:49.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.440+0000 7fa6b502d700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa69c03e570 0x7fa69c040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006fd0 tx=0x7fa6a0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:49.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.445+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6ac013070 con 0x7fa6b0103a50 2024-05-08T05:42:49.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.665+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa6b0191820 con 0x7fa69c03e570 2024-05-08T05:42:49.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.669+0000 7fa6a6ffd700 1 -- 172.21.15.3:0/499956855 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa6b0191820 con 0x7fa69c03e570 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa69c03e570 msgr2=0x7fa69c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa69c03e570 0x7fa69c040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0006fd0 tx=0x7fa6a0006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 msgr2=0x7fa6b0103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0103280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac005b40 tx=0x7fa6ac004f80 comp rx=0 tx=0).stop 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 shutdown_connections 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa69c03e570 0x7fa69c040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 --2- 172.21.15.3:0/499956855 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa6b0103a50 0x7fa6b0103280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 >> 172.21.15.3:0/499956855 conn(0x7fa6b00fd3a0 msgr2=0x7fa6b00fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 shutdown_connections 2024-05-08T05:42:49.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:49.671+0000 7fa6b7a92700 1 -- 172.21.15.3:0/499956855 wait complete. 2024-05-08T05:42:50.370 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi089 2024-05-08T05:42:50.370 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:42:50.370 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:42:50.389 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:42:50.389 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:42:50.446 INFO:tasks.cephadm:Adding host smithi089 to orchestrator... 2024-05-08T05:42:50.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch host add smithi089 2024-05-08T05:42:50.722 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:42:50.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:50.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:50.949 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:50.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:42:50.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:42:50.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:42:50.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:42:50.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:42:50.950 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:50 smithi003 ceph-mon[30137]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:42:51.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:51.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:51.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 -- 172.21.15.3:0/3702115391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 msgr2=0x7f286c104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 --2- 172.21.15.3:0/3702115391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c104490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f285c009910 tx=0x7f285c009c20 comp rx=0 tx=0).stop 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 -- 172.21.15.3:0/3702115391 shutdown_connections 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 --2- 172.21.15.3:0/3702115391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c104490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 -- 172.21.15.3:0/3702115391 >> 172.21.15.3:0/3702115391 conn(0x7f286c0fd900 msgr2=0x7f286c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 -- 172.21.15.3:0/3702115391 shutdown_connections 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 -- 172.21.15.3:0/3702115391 wait complete. 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.178+0000 7f2872356700 1 Processor -- start 2024-05-08T05:42:52.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f2872356700 1 -- start start 2024-05-08T05:42:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f2872356700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f2872356700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10d430 con 0x7f286c1040b0 2024-05-08T05:42:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f286bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f286bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c1a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52534/0 (socket says 172.21.15.3:52534) 2024-05-08T05:42:52.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f286bfff700 1 -- 172.21.15.3:0/1091110394 learned_addr learned my addr 172.21.15.3:0/1091110394 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.179+0000 7f286bfff700 1 -- 172.21.15.3:0/1091110394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c1a0d90 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.180+0000 7f286bfff700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c1a0850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f285c0056d0 tx=0x7f285c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.180+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f285c026070 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.180+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f285c004370 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.180+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c1a1020 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.180+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f285c0039b0 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.180+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c19aa10 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.181+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f285c029070 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.181+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c06bbf0 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.181+0000 7f28697fa700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f285403e570 0x7f2854040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.181+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f285c01e070 con 0x7f286c1040b0 2024-05-08T05:42:52.182 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.181+0000 7f286b7fe700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f285403e570 0x7f2854040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:52.183 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.182+0000 7f286b7fe700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f285403e570 0x7f2854040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2860006fd0 tx=0x7f2860006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:52.186 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.186+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f285c013070 con 0x7f286c1040b0 2024-05-08T05:42:52.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:52.404+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}) -- 0x7f286c19bf40 con 0x7f285403e570 2024-05-08T05:42:53.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:52 smithi003 ceph-mon[30137]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi089", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:42:54.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:54 smithi003 ceph-mon[30137]: Deploying cephadm binary to smithi089 2024-05-08T05:42:54.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:54 smithi003 ceph-mon[30137]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:42:56.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:56 smithi003 ceph-mon[30137]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:42:56.746 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi089' with addr '172.21.15.89' 2024-05-08T05:42:56.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.745+0000 7f28697fa700 1 -- 172.21.15.3:0/1091110394 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f286c19bf40 con 0x7f285403e570 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.748+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f285403e570 msgr2=0x7f2854040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.748+0000 7f2872356700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f285403e570 0x7f2854040a30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2860006fd0 tx=0x7f2860006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.748+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 msgr2=0x7f286c1a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.748+0000 7f2872356700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c1a0850 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f285c0056d0 tx=0x7f285c0057b0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.749+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 shutdown_connections 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.749+0000 7f2872356700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f285403e570 0x7f2854040a30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.749+0000 7f2872356700 1 --2- 172.21.15.3:0/1091110394 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f286c1040b0 0x7f286c1a0850 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.749+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 >> 172.21.15.3:0/1091110394 conn(0x7f286c0fd900 msgr2=0x7f286c109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.749+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 shutdown_connections 2024-05-08T05:42:56.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:56.749+0000 7f2872356700 1 -- 172.21.15.3:0/1091110394 wait complete. 2024-05-08T05:42:57.370 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:42:57.621 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:42:58.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:58.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:57 smithi003 ceph-mon[30137]: Added host smithi089 2024-05-08T05:42:58.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:42:58.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:57 smithi003 ceph-mon[30137]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:42:58.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:59.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:42:58 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 -- 172.21.15.3:0/2204953952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 msgr2=0x7fe9f01082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 --2- 172.21.15.3:0/2204953952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01082d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009910 tx=0x7fe9e0009c20 comp rx=0 tx=0).stop 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 -- 172.21.15.3:0/2204953952 shutdown_connections 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 --2- 172.21.15.3:0/2204953952 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01082d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 -- 172.21.15.3:0/2204953952 >> 172.21.15.3:0/2204953952 conn(0x7fe9f00fd050 msgr2=0x7fe9f00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 -- 172.21.15.3:0/2204953952 shutdown_connections 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.053+0000 7fe9f622b700 1 -- 172.21.15.3:0/2204953952 wait complete. 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.054+0000 7fe9f622b700 1 Processor -- start 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.054+0000 7fe9f622b700 1 -- start start 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9f622b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01024f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9f622b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f010d500 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01024f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58922/0 (socket says 172.21.15.3:58922) 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ef7fe700 1 -- 172.21.15.3:0/2130501352 learned_addr learned my addr 172.21.15.3:0/2130501352 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ef7fe700 1 -- 172.21.15.3:0/2130501352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f0104340 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ef7fe700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01024f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00056d0 tx=0x7fe9e00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0026070 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f0102a30 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9e000bc00 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9e0003920 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.055+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f0102e30 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.056+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe9e0029070 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.056+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f0064b00 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.056+0000 7fe9ed7fa700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9dc03e510 0x7fe9dc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.057+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe9e001e070 con 0x7fe9f0107ef0 2024-05-08T05:42:59.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.057+0000 7fe9e6ffd700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9dc03e510 0x7fe9dc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:42:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.057+0000 7fe9e6ffd700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9dc03e510 0x7fe9dc0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fd0 tx=0x7fe9d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:42:59.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.062+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9e0013070 con 0x7fe9f0107ef0 2024-05-08T05:42:59.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.286+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9f0103060 con 0x7fe9dc03e510 2024-05-08T05:42:59.287 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.287+0000 7fe9ed7fa700 1 -- 172.21.15.3:0/2130501352 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7fe9f0103060 con 0x7fe9dc03e510 2024-05-08T05:42:59.288 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:42:59.288 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}] 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.289+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9dc03e510 msgr2=0x7fe9dc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.289+0000 7fe9f622b700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9dc03e510 0x7fe9dc0409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8006fd0 tx=0x7fe9d8006e40 comp rx=0 tx=0).stop 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.289+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 msgr2=0x7fe9f01024f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.289+0000 7fe9f622b700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01024f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00056d0 tx=0x7fe9e00057b0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.290+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 shutdown_connections 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.290+0000 7fe9f622b700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9dc03e510 0x7fe9dc0409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.290+0000 7fe9f622b700 1 --2- 172.21.15.3:0/2130501352 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f0107ef0 0x7fe9f01024f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:42:59.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.290+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 >> 172.21.15.3:0/2130501352 conn(0x7fe9f00fd050 msgr2=0x7fe9f010ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:42:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.290+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 shutdown_connections 2024-05-08T05:42:59.291 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:42:59.290+0000 7fe9f622b700 1 -- 172.21.15.3:0/2130501352 wait complete. 2024-05-08T05:42:59.801 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-05-08T05:42:59.802 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:42:59.802 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:42:59.820 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:42:59.820 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:42:59.877 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-05-08T05:42:59.877 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch host add smithi104 2024-05-08T05:43:00.121 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:00.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:00 smithi003 ceph-mon[30137]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:00.579 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:00 smithi003 ceph-mon[30137]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:00.580 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:01.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 -- 172.21.15.3:0/960984970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 msgr2=0x7fafa01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:01.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 --2- 172.21.15.3:0/960984970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa01023c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf90009910 tx=0x7faf90009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 -- 172.21.15.3:0/960984970 shutdown_connections 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 --2- 172.21.15.3:0/960984970 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa01023c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 -- 172.21.15.3:0/960984970 >> 172.21.15.3:0/960984970 conn(0x7fafa00fd8b0 msgr2=0x7fafa00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 -- 172.21.15.3:0/960984970 shutdown_connections 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.794+0000 7fafa666a700 1 -- 172.21.15.3:0/960984970 wait complete. 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7fafa666a700 1 Processor -- start 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7fafa666a700 1 -- start start 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7fafa666a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7fafa666a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fafa010c260 con 0x7fafa0101fe0 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7faf9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7faf9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa0199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58940/0 (socket says 172.21.15.3:58940) 2024-05-08T05:43:01.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.795+0000 7faf9ffff700 1 -- 172.21.15.3:0/795049346 learned_addr learned my addr 172.21.15.3:0/795049346 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:43:01.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7faf9ffff700 1 -- 172.21.15.3:0/795049346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa0199ca0 con 0x7fafa0101fe0 2024-05-08T05:43:01.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7faf9ffff700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa0199760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf900056d0 tx=0x7faf900057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:01.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf90026070 con 0x7fafa0101fe0 2024-05-08T05:43:01.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9000bba0 con 0x7fafa0101fe0 2024-05-08T05:43:01.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa0199f90 con 0x7fafa0101fe0 2024-05-08T05:43:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf90003920 con 0x7fafa0101fe0 2024-05-08T05:43:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.796+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa019a450 con 0x7fafa0101fe0 2024-05-08T05:43:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.797+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa01034c0 con 0x7fafa0101fe0 2024-05-08T05:43:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.797+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7faf90029070 con 0x7fafa0101fe0 2024-05-08T05:43:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.797+0000 7faf9d7fa700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7faf8803e510 0x7faf880409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:01.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.797+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faf9001e070 con 0x7fafa0101fe0 2024-05-08T05:43:01.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.798+0000 7faf9f7fe700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7faf8803e510 0x7faf880409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:01.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.798+0000 7faf9f7fe700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7faf8803e510 0x7faf880409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf94006fd0 tx=0x7faf94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:01.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:01.802+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf90013070 con 0x7fafa0101fe0 2024-05-08T05:43:02.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:02.025+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) -- 0x7fafa006b8c0 con 0x7faf8803e510 2024-05-08T05:43:02.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:02 smithi003 ceph-mon[30137]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:43:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:03 smithi003 ceph-mon[30137]: Deploying cephadm binary to smithi104 2024-05-08T05:43:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:04 smithi003 ceph-mon[30137]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:04 smithi003 ceph-mon[30137]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:04 smithi003 ceph-mon[30137]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:06.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.236+0000 7faf9d7fa700 1 -- 172.21.15.3:0/795049346 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fafa006b8c0 con 0x7faf8803e510 2024-05-08T05:43:06.237 INFO:teuthology.orchestra.run.smithi003.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7faf8803e510 msgr2=0x7faf880409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7faf8803e510 0x7faf880409d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf94006fd0 tx=0x7faf94006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 msgr2=0x7fafa0199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa0199760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf900056d0 tx=0x7faf900057b0 comp rx=0 tx=0).stop 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 shutdown_connections 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7faf8803e510 0x7faf880409d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 --2- 172.21.15.3:0/795049346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fafa0101fe0 0x7fafa0199760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 >> 172.21.15.3:0/795049346 conn(0x7fafa00fd8b0 msgr2=0x7fafa01039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 shutdown_connections 2024-05-08T05:43:06.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:06.238+0000 7fafa666a700 1 -- 172.21.15.3:0/795049346 wait complete. 2024-05-08T05:43:06.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:06.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:06.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:06.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:06.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:06.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:06.770 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:06.800 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:43:07.044 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:07.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:07 smithi003 ceph-mon[30137]: Added host smithi104 2024-05-08T05:43:07.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:07.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:08.628 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:08 smithi003 ceph-mon[30137]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.659+0000 7fa0b3046700 1 -- 172.21.15.3:0/2083829461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 msgr2=0x7fa0ac1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.659+0000 7fa0b3046700 1 --2- 172.21.15.3:0/2083829461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac1034f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009910 tx=0x7fa0a0009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 -- 172.21.15.3:0/2083829461 shutdown_connections 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 --2- 172.21.15.3:0/2083829461 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac1034f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 -- 172.21.15.3:0/2083829461 >> 172.21.15.3:0/2083829461 conn(0x7fa0ac0fe9e0 msgr2=0x7fa0ac100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 -- 172.21.15.3:0/2083829461 shutdown_connections 2024-05-08T05:43:08.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 -- 172.21.15.3:0/2083829461 wait complete. 2024-05-08T05:43:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 Processor -- start 2024-05-08T05:43:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 -- start start 2024-05-08T05:43:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.660+0000 7fa0b3046700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac10d390 con 0x7fa0ac103110 2024-05-08T05:43:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b0de2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b0de2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53774/0 (socket says 172.21.15.3:53774) 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b0de2700 1 -- 172.21.15.3:0/2172376120 learned_addr learned my addr 172.21.15.3:0/2172376120 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b0de2700 1 -- 172.21.15.3:0/2172376120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac19eb70 con 0x7fa0ac103110 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b0de2700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac19e630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006010 tx=0x7fa0a0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0026070 con 0x7fa0ac103110 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0a000bba0 con 0x7fa0ac103110 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac19ee60 con 0x7fa0ac103110 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0a0003910 con 0x7fa0ac103110 2024-05-08T05:43:08.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.661+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac1988d0 con 0x7fa0ac103110 2024-05-08T05:43:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.662+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fa0a0029070 con 0x7fa0ac103110 2024-05-08T05:43:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.662+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ac1045f0 con 0x7fa0ac103110 2024-05-08T05:43:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.663+0000 7fa0a9ffb700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa09c03e570 0x7fa09c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.663+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa0a001e070 con 0x7fa0ac103110 2024-05-08T05:43:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.663+0000 7fa0abfff700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa09c03e570 0x7fa09c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:08.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.663+0000 7fa0abfff700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa09c03e570 0x7fa09c040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa098006fd0 tx=0x7fa098006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:08.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.668+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0a0013070 con 0x7fa0ac103110 2024-05-08T05:43:08.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.899+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0ac199c60 con 0x7fa09c03e570 2024-05-08T05:43:08.900 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.900+0000 7fa0a9ffb700 1 -- 172.21.15.3:0/2172376120 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7fa0ac199c60 con 0x7fa09c03e570 2024-05-08T05:43:08.901 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:43:08.901 INFO:teuthology.orchestra.run.smithi003.stdout:[{"addr": "172.21.15.3", "hostname": "smithi003", "labels": [], "status": ""}, {"addr": "172.21.15.89", "hostname": "smithi089", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-05-08T05:43:08.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa09c03e570 msgr2=0x7fa09c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa09c03e570 0x7fa09c040a30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa098006fd0 tx=0x7fa098006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 msgr2=0x7fa0ac19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac19e630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006010 tx=0x7fa0a0005780 comp rx=0 tx=0).stop 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 shutdown_connections 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fa09c03e570 0x7fa09c040a30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 --2- 172.21.15.3:0/2172376120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa0ac103110 0x7fa0ac19e630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 >> 172.21.15.3:0/2172376120 conn(0x7fa0ac0fe9e0 msgr2=0x7fa0ac104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 shutdown_connections 2024-05-08T05:43:08.904 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:08.903+0000 7fa0b3046700 1 -- 172.21.15.3:0/2172376120 wait complete. 2024-05-08T05:43:09.479 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T05:43:09.479 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T05:43:09.728 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:10.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:09 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:10.094 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:09 smithi003 ceph-mon[30137]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:10.094 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:09 smithi003 ceph-mon[30137]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 -- 172.21.15.3:0/2144436132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 msgr2=0x7f5628104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 --2- 172.21.15.3:0/2144436132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f5628104450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5624009910 tx=0x7f5624009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 -- 172.21.15.3:0/2144436132 shutdown_connections 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 --2- 172.21.15.3:0/2144436132 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f5628104450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 -- 172.21.15.3:0/2144436132 >> 172.21.15.3:0/2144436132 conn(0x7f56280fd8a0 msgr2=0x7f56280ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 -- 172.21.15.3:0/2144436132 shutdown_connections 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.503+0000 7f562f9db700 1 -- 172.21.15.3:0/2144436132 wait complete. 2024-05-08T05:43:11.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562f9db700 1 Processor -- start 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562f9db700 1 -- start start 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562f9db700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f562819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562f9db700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f562810d3f0 con 0x7f5628104070 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562d777700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f562819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562d777700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f562819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53786/0 (socket says 172.21.15.3:53786) 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562d777700 1 -- 172.21.15.3:0/1408479612 learned_addr learned my addr 172.21.15.3:0/1408479612 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562d777700 1 -- 172.21.15.3:0/1408479612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562819ebe0 con 0x7f5628104070 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.504+0000 7f562d777700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f562819e6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56240056d0 tx=0x7f56240057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.505+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5624026070 con 0x7f5628104070 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.505+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562819ee70 con 0x7f5628104070 2024-05-08T05:43:11.506 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.505+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5624004370 con 0x7f5628104070 2024-05-08T05:43:11.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.505+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56240039b0 con 0x7f5628104070 2024-05-08T05:43:11.507 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.505+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5628198770 con 0x7f5628104070 2024-05-08T05:43:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.506+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f5624029070 con 0x7f5628104070 2024-05-08T05:43:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.506+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562806bb90 con 0x7f5628104070 2024-05-08T05:43:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.506+0000 7f561e7fc700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f561403e510 0x7f56140409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.506+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f562401e070 con 0x7f5628104070 2024-05-08T05:43:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.506+0000 7f562cf76700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f561403e510 0x7f56140409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:11.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.506+0000 7f562cf76700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f561403e510 0x7f56140409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5618006fd0 tx=0x7f5618006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:11.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.511+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5624013070 con 0x7f5628104070 2024-05-08T05:43:11.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:11.731+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5628064b00 con 0x7f5628104070 2024-05-08T05:43:12.184 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.183+0000 7f561e7fc700 1 -- 172.21.15.3:0/1408479612 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5624027260 con 0x7f5628104070 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f561403e510 msgr2=0x7f56140409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f561403e510 0x7f56140409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5618006fd0 tx=0x7f5618006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 msgr2=0x7f562819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f562819e6a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f56240056d0 tx=0x7f56240057b0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 shutdown_connections 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f561403e510 0x7f56140409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 --2- 172.21.15.3:0/1408479612 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5628104070 0x7f562819e6a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 >> 172.21.15.3:0/1408479612 conn(0x7f56280fd8a0 msgr2=0x7f5628104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 shutdown_connections 2024-05-08T05:43:12.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:12.186+0000 7f562f9db700 1 -- 172.21.15.3:0/1408479612 wait complete. 2024-05-08T05:43:12.188 INFO:teuthology.orchestra.run.smithi003.stderr:adjusted tunables profile to default 2024-05-08T05:43:12.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:12.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1408479612' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:43:12.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:12.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:12.881 INFO:tasks.cephadm:Adding mon.a on smithi003 2024-05-08T05:43:12.881 INFO:tasks.cephadm:Adding mon.b on smithi089 2024-05-08T05:43:12.881 INFO:tasks.cephadm:Adding mon.c on smithi104 2024-05-08T05:43:12.881 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c' 2024-05-08T05:43:13.109 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:43:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1408479612' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:43:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:13.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:13.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:13.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:14.508 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 -- 172.21.15.104:0/1009556591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 msgr2=0x7f91841034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 --2- 172.21.15.104:0/1009556591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91841034e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9174009910 tx=0x7f9174009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 -- 172.21.15.104:0/1009556591 shutdown_connections 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 --2- 172.21.15.104:0/1009556591 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91841034e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 -- 172.21.15.104:0/1009556591 >> 172.21.15.104:0/1009556591 conn(0x7f91840fe990 msgr2=0x7f9184100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 -- 172.21.15.104:0/1009556591 shutdown_connections 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.507+0000 7f918a919700 1 -- 172.21.15.104:0/1009556591 wait complete. 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.508+0000 7f918a919700 1 Processor -- start 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.508+0000 7f918a919700 1 -- start start 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.508+0000 7f918a919700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91840757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:14.509 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.508+0000 7f918a919700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f918410d380 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f9183fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91840757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f9183fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91840757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.104:44120/0 (socket says 172.21.15.104:44120) 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f9183fff700 1 -- 172.21.15.104:0/2993173863 learned_addr learned my addr 172.21.15.104:0/2993173863 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f9183fff700 1 -- 172.21.15.104:0/2993173863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9184077640 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f9183fff700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91840757a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f91740056d0 tx=0x7f91740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9174026070 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.509+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9184075ce0 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.510+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f917400bc00 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.510+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9174003920 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.510+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91840761a0 con 0x7f9184103100 2024-05-08T05:43:14.511 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.510+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f9174029070 con 0x7f9184103100 2024-05-08T05:43:14.512 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.511+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91841045e0 con 0x7f9184103100 2024-05-08T05:43:14.512 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.511+0000 7f91817fa700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f916c03e510 0x7f916c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:14.512 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.511+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f917401e070 con 0x7f9184103100 2024-05-08T05:43:14.512 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.511+0000 7f91837fe700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f916c03e510 0x7f916c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:14.512 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.511+0000 7f91837fe700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f916c03e510 0x7f916c0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9178006fd0 tx=0x7f9178006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:14.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.516+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9174013070 con 0x7f9184103100 2024-05-08T05:43:14.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:14 smithi003 ceph-mon[30137]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:14.733 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.733+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}) -- 0x7f9184076cf0 con 0x7f916c03e510 2024-05-08T05:43:14.740 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.740+0000 7f91817fa700 1 -- 172.21.15.104:0/2993173863 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9184076cf0 con 0x7f916c03e510 2024-05-08T05:43:14.740 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mon update... 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f916c03e510 msgr2=0x7f916c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f916c03e510 0x7f916c0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9178006fd0 tx=0x7f9178006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 msgr2=0x7f91840757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91840757a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f91740056d0 tx=0x7f91740057b0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 shutdown_connections 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f916c03e510 0x7f916c0409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 --2- 172.21.15.104:0/2993173863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9184103100 0x7f91840757a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 >> 172.21.15.104:0/2993173863 conn(0x7f91840fe990 msgr2=0x7f9184104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 shutdown_connections 2024-05-08T05:43:14.743 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:14.742+0000 7f918a919700 1 -- 172.21.15.104:0/2993173863 wait complete. 2024-05-08T05:43:15.319 DEBUG:teuthology.orchestra.run.smithi089:mon.b> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T05:43:15.322 DEBUG:teuthology.orchestra.run.smithi104:mon.c> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T05:43:15.324 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:43:15.324 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:43:15.964 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:16.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:16.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c;count:3 2024-05-08T05:43:16.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:16.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:16.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:16.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:16.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:16.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:16.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:16.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: Deploying daemon mon.c on smithi104 2024-05-08T05:43:16.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:15 smithi003 ceph-mon[30137]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:18.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:18 smithi003 ceph-mon[30137]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 -- 172.21.15.104:0/1051656058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 msgr2=0x7fe018102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 --2- 172.21.15.104:0/1051656058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe018102430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe008009910 tx=0x7fe008009c20 comp rx=0 tx=0).stop 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 -- 172.21.15.104:0/1051656058 shutdown_connections 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 --2- 172.21.15.104:0/1051656058 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe018102430 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 -- 172.21.15.104:0/1051656058 >> 172.21.15.104:0/1051656058 conn(0x7fe018077900 msgr2=0x7fe018077d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 -- 172.21.15.104:0/1051656058 shutdown_connections 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.047+0000 7fe01fef3700 1 -- 172.21.15.104:0/1051656058 wait complete. 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01fef3700 1 Processor -- start 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01fef3700 1 -- start start 2024-05-08T05:43:19.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01fef3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe0181996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:19.050 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01fef3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe01810c280 con 0x7fe018102050 2024-05-08T05:43:19.050 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01dc8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe0181996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:19.050 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01dc8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe0181996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.104:37254/0 (socket says 172.21.15.104:37254) 2024-05-08T05:43:19.050 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.048+0000 7fe01dc8f700 1 -- 172.21.15.104:0/2741547609 learned_addr learned my addr 172.21.15.104:0/2741547609 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:43:19.050 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe01dc8f700 1 -- 172.21.15.104:0/2741547609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe018199c30 con 0x7fe018102050 2024-05-08T05:43:19.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe01dc8f700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe0181996f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0080056d0 tx=0x7fe0080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:19.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008026070 con 0x7fe018102050 2024-05-08T05:43:19.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00800bba0 con 0x7fe018102050 2024-05-08T05:43:19.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe018199f20 con 0x7fe018102050 2024-05-08T05:43:19.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008003920 con 0x7fe018102050 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.049+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01819a3e0 con 0x7fe018102050 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.050+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe008029070 con 0x7fe018102050 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.050+0000 7fe00effd700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe00403e510 0x7fe0040409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.050+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0181034e0 con 0x7fe018102050 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.051+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe00801e070 con 0x7fe018102050 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.051+0000 7fe01d48e700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe00403e510 0x7fe0040409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:19.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.051+0000 7fe01d48e700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe00403e510 0x7fe0040409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe014006fd0 tx=0x7fe014006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:19.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.056+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe008013070 con 0x7fe018102050 2024-05-08T05:43:19.390 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.390+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe018064b00 con 0x7fe018102050 2024-05-08T05:43:19.391 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.390+0000 7fe00effd700 1 -- 172.21.15.104:0/2741547609 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fe008030020 con 0x7fe018102050 2024-05-08T05:43:19.391 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:43:19.392 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","modified":"2024-05-08T05:41:30.871663Z","created":"2024-05-08T05:41:30.871663Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.392+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe00403e510 msgr2=0x7fe0040409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.392+0000 7fe01fef3700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe00403e510 0x7fe0040409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe014006fd0 tx=0x7fe014006e40 comp rx=0 tx=0).stop 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 msgr2=0x7fe0181996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe0181996f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0080056d0 tx=0x7fe0080057b0 comp rx=0 tx=0).stop 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 shutdown_connections 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe00403e510 0x7fe0040409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 --2- 172.21.15.104:0/2741547609 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe018102050 0x7fe0181996f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 >> 172.21.15.104:0/2741547609 conn(0x7fe018077900 msgr2=0x7fe018109b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:19.393 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 shutdown_connections 2024-05-08T05:43:19.394 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:19.393+0000 7fe01fef3700 1 -- 172.21.15.104:0/2741547609 wait complete. 2024-05-08T05:43:19.394 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-05-08T05:43:19.503 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 podman[34528]: 2024-05-08 05:43:19.503718218 +0000 UTC m=+0.140685523 container create 8f7693d5b131b0f4f4f7c7310dffec012fe1eb9f21366f4cf9b393b4d8a4ab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-08T05:43:20.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 podman[34528]: 2024-05-08 05:43:19.882137155 +0000 UTC m=+0.519104455 container init 8f7693d5b131b0f4f4f7c7310dffec012fe1eb9f21366f4cf9b393b4d8a4ab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-08T05:43:20.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 podman[34528]: 2024-05-08 05:43:19.886001603 +0000 UTC m=+0.522968903 container start 8f7693d5b131b0f4f4f7c7310dffec012fe1eb9f21366f4cf9b393b4d8a4ab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-08T05:43:20.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:43:20.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:43:20.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pidfile_write: ignore empty --pid-file 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: load: jerasure load: lrc 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Git sha 0 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: DB SUMMARY 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: DB Session ID: RKIE6AUX8YAAEUOPKYOM 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-08T05:43:20.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.env: 0x562794bc3cc0 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.info_log: 0x5627956f1da0 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.statistics: (nil) 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.use_fsync: 0 2024-05-08T05:43:20.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.db_log_dir: 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.wal_dir: 2024-05-08T05:43:20.275 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.write_buffer_manager: 0x5627956c30e0 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:43:20.276 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.unordered_write: 0 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.row_cache: None 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.wal_filter: None 2024-05-08T05:43:20.277 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.wal_compression: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:43:20.278 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:43:20.279 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_open_files: -1 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Compression algorithms supported: 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:43:20.280 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kZSTD supported: 0 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:43:20.281 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.merge_operator: 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_filter: None 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5627956f1f60) 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: index_type: 0 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: data_block_index_type: 0 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: index_shortening: 1 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:43:20.282 INFO:journalctl@ceph.mon.c.smithi104.stdout: checksum: 4 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: no_block_cache: 0 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_cache: 0x5627956f51f0 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_cache_options: 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: capacity : 536870912 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: num_shard_bits : 4 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: strict_capacity_limit : 0 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_cache_compressed: (nil) 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: persistent_cache: (nil) 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_size: 4096 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_size_deviation: 10 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_restart_interval: 16 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: index_block_restart_interval: 1 2024-05-08T05:43:20.283 INFO:journalctl@ceph.mon.c.smithi104.stdout: metadata_block_size: 4096 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: partition_filters: 0 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: use_delta_encoding: 1 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: filter_policy: bloomfilter 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: whole_key_filtering: 1 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: verify_compression: 0 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: format_version: 5 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: enable_index_compression: 1 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: block_align: 0 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: max_auto_readahead_size: 262144 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: prepopulate_block_cache: 0 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:43:20.284 INFO:journalctl@ceph.mon.c.smithi104.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:43:20.285 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression: NoCompression 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.num_levels: 7 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:43:20.286 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:43:20.287 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:43:20.288 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:43:20.289 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:43:20.290 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.ttl: 2592000 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:43:20.291 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 19f87187-498b-4d65-af5b-e203ed660ca9 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146999915701, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146999916165, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715146999, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "19f87187-498b-4d65-af5b-e203ed660ca9", "db_session_id": "RKIE6AUX8YAAEUOPKYOM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715146999916244, "job": 1, "event": "recovery_finished"} 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562795812000 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: DB pointer 0x5627957fe000 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout: ** DB Stats ** 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:43:20.292 INFO:journalctl@ceph.mon.c.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: ** Compaction Stats [default] ** 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:43:20.293 INFO:journalctl@ceph.mon.c.smithi104.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: ** Compaction Stats [default] ** 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:43:20.294 INFO:journalctl@ceph.mon.c.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: Block cache BinnedLRUCache@0x5627956f51f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: using public_addr v2:172.21.15.104:0/0 -> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: starting mon.c rank -1 at public addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] at bind addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(???) e0 preinit fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:20.295 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).mds e1 new map 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout: e1 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout: legacy client fscid: -1 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout: 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout: No filesystems configured 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:43:20.296 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi089", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:20.297 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Deploying cephadm binary to smithi104 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Added host smithi104 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.298 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1408479612' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:43:20.299 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1408479612' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Saving service mon spec with placement smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c;count:3 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:20.300 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: Deploying daemon mon.c on smithi104 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 ceph-mon[34559]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 bash[34528]: 8f7693d5b131b0f4f4f7c7310dffec012fe1eb9f21366f4cf9b393b4d8a4ab39 2024-05-08T05:43:20.301 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:19 smithi104 systemd[1]: Started Ceph mon.c for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:43:21.507 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:43:21.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:43:21.747 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: mon.a calling monitor election 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: mon.c calling monitor election 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: monmap epoch 2 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: last_changed 2024-05-08T05:43:19.939289+0000 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: min_mon_release 19 (squid) 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: election_strategy: 1 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: fsmap 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: mgrmap e13: x(active, since 51s) 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: overall HEALTH_OK 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:25.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:25.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:25.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:25.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:25.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:43:25.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: mon.a calling monitor election 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: mon.c calling monitor election 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: monmap epoch 2 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: last_changed 2024-05-08T05:43:19.939289+0000 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:43:25.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: min_mon_release 19 (squid) 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: election_strategy: 1 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: fsmap 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: mgrmap e13: x(active, since 51s) 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: overall HEALTH_OK 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:25.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:25.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:26.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.051+0000 7f316d070700 1 -- 172.21.15.104:0/3841149354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31540035c0 msgr2=0x7f3154005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:26.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.052+0000 7f316d070700 1 --2- 172.21.15.104:0/3841149354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31540035c0 0x7f3154005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3168101dc0 tx=0x7f315800feb0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.052+0000 7f316d070700 1 -- 172.21.15.104:0/3841149354 shutdown_connections 2024-05-08T05:43:26.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.052+0000 7f316d070700 1 --2- 172.21.15.104:0/3841149354 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31540035c0 0x7f3154005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.052+0000 7f316d070700 1 -- 172.21.15.104:0/3841149354 >> 172.21.15.104:0/3841149354 conn(0x7f31680fd9c0 msgr2=0x7f31680ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:26.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.052+0000 7f316d070700 1 -- 172.21.15.104:0/3841149354 shutdown_connections 2024-05-08T05:43:26.053 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.052+0000 7f316d070700 1 -- 172.21.15.104:0/3841149354 wait complete. 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.053+0000 7f316d070700 1 Processor -- start 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.053+0000 7f316d070700 1 -- start start 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f316d070700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f316d070700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 0x7f316819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f316d070700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3168109de0 con 0x7f31681a0dd0 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f316d070700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3168109c60 con 0x7f31540035c0 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 0x7f316819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f3166d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 0x7f316819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.104:37298/0 (socket says 172.21.15.104:37298) 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 -- 172.21.15.104:0/346066733 learned_addr learned my addr 172.21.15.104:0/346066733 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:43:26.055 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f3166d9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:40906/0 (socket says 172.21.15.104:40906) 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f3166d9d700 1 -- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 msgr2=0x7f31681a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f3166d9d700 1 -- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 msgr2=0x7f31681a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f3166d9d700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f3166d9d700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 -- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 msgr2=0x7f31681a0890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 -- 172.21.15.104:0/346066733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31681a1410 con 0x7f31681a0dd0 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.054+0000 7f315e59c700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 0x7f316819aa50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f315000b700 tx=0x7f315000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.055+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3150011630 con 0x7f31681a0dd0 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.055+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3150011c80 con 0x7f31681a0dd0 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.055+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f316819af90 con 0x7f31681a0dd0 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.055+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f3150010da0 con 0x7f31681a0dd0 2024-05-08T05:43:26.056 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.055+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316819b520 con 0x7f31681a0dd0 2024-05-08T05:43:26.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.056+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f3150018020 con 0x7f31681a0dd0 2024-05-08T05:43:26.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.056+0000 7f3164d99700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f314803e300 0x7f31480407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:26.057 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.056+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f31500530d0 con 0x7f31681a0dd0 2024-05-08T05:43:26.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.056+0000 7f3166d9d700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f314803e300 0x7f31480407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:26.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.056+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31681035d0 con 0x7f31681a0dd0 2024-05-08T05:43:26.058 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.057+0000 7f3166d9d700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f314803e300 0x7f31480407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31580059d0 tx=0x7f3158005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:26.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.062+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f315001f020 con 0x7f31681a0dd0 2024-05-08T05:43:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:26 smithi104 ceph-mon[34559]: Deploying daemon mon.b on smithi089 2024-05-08T05:43:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:26 smithi104 ceph-mon[34559]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:26.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.351+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f316819bf80 con 0x7f31681a0dd0 2024-05-08T05:43:26.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.352+0000 7f3164d99700 1 -- 172.21.15.104:0/346066733 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f31500187b0 con 0x7f31681a0dd0 2024-05-08T05:43:26.352 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:43:26.352 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":2,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","modified":"2024-05-08T05:43:19.939289Z","created":"2024-05-08T05:41:30.871663Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f314803e300 msgr2=0x7f31480407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f314803e300 0x7f31480407c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31580059d0 tx=0x7f3158005960 comp rx=0 tx=0).stop 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 msgr2=0x7f316819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 0x7f316819aa50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f315000b700 tx=0x7f315000bac0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 shutdown_connections 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31540035c0 0x7f31681a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f314803e300 0x7f31480407c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 --2- 172.21.15.104:0/346066733 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f31681a0dd0 0x7f316819aa50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:26.355 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 >> 172.21.15.104:0/346066733 conn(0x7f31680fd9c0 msgr2=0x7f31680ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:26.355 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 shutdown_connections 2024-05-08T05:43:26.355 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:26.354+0000 7f316d070700 1 -- 172.21.15.104:0/346066733 wait complete. 2024-05-08T05:43:26.355 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 2 2024-05-08T05:43:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:26 smithi003 ceph-mon[30137]: Deploying daemon mon.b on smithi089 2024-05-08T05:43:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:26 smithi003 ceph-mon[30137]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:27.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:27 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/346066733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:43:27.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:27 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/346066733' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:43:27.410 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:43:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:43:26.939+0000 7f0aa7299700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T05:43:27.969 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:43:27.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:43:28.214 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:43:28.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:28 smithi104 ceph-mon[34559]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:28.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:28 smithi003 ceph-mon[30137]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:29.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:29 smithi089 systemd[1]: Started Ceph mon.b for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:43:34.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:43:34.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: mon.a calling monitor election 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: mon.c calling monitor election 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: mon.b calling monitor election 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:43:34.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: monmap epoch 3 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: last_changed 2024-05-08T05:43:29.008504+0000 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: min_mon_release 19 (squid) 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: election_strategy: 1 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: fsmap 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: mgrmap e13: x(active, since 60s) 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: overall HEALTH_OK 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: mon.a calling monitor election 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: mon.c calling monitor election 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: mon.b calling monitor election 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: monmap epoch 3 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: last_changed 2024-05-08T05:43:29.008504+0000 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: min_mon_release 19 (squid) 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: election_strategy: 1 2024-05-08T05:43:34.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: fsmap 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: mgrmap e13: x(active, since 60s) 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: overall HEALTH_OK 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: mon.a calling monitor election 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: mon.c calling monitor election 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: mon.b calling monitor election 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: monmap epoch 3 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: last_changed 2024-05-08T05:43:29.008504+0000 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: created 2024-05-08T05:41:30.871663+0000 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: min_mon_release 19 (squid) 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: election_strategy: 1 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: 0: [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] mon.a 2024-05-08T05:43:34.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.c 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: 2: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.b 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: fsmap 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: mgrmap e13: x(active, since 60s) 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: overall HEALTH_OK 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:34.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:35.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:35 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:35.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:35 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:35.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:35 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:35 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:35.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:35 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:35.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:35 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.569+0000 7f354c597700 1 -- 172.21.15.104:0/2387718998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 msgr2=0x7f35300086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.569+0000 7f354c597700 1 --2- 172.21.15.104:0/2387718998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f35300086a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f354406b910 tx=0x7f354000fe90 comp rx=0 tx=0).stop 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- 172.21.15.104:0/2387718998 shutdown_connections 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 --2- 172.21.15.104:0/2387718998 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f35300086a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- 172.21.15.104:0/2387718998 >> 172.21.15.104:0/2387718998 conn(0x7f35440fd9e0 msgr2=0x7f35440ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- 172.21.15.104:0/2387718998 shutdown_connections 2024-05-08T05:43:35.572 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- 172.21.15.104:0/2387718998 wait complete. 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 Processor -- start 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- start start 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f354419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f354419f7c0 0x7f35441a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35441a4090 0x7f35441a4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f35441114a0 con 0x7f3530006230 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3544111320 con 0x7f35441a4090 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.570+0000 7f354c597700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3544111620 con 0x7f354419f7c0 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f3549b32700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f354419f7c0 0x7f35441a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f354419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f3549b32700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f354419f7c0 0x7f35441a3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49656/0 (socket says 172.21.15.104:49656) 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f354419f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.104:36940/0 (socket says 172.21.15.104:36940) 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f3549b32700 1 -- 172.21.15.104:0/609415611 learned_addr learned my addr 172.21.15.104:0/609415611 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:43:35.573 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354ab34700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35441a4090 0x7f35441a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 -- 172.21.15.104:0/609415611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f354419f7c0 msgr2=0x7f35441a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f354419f7c0 0x7f35441a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 -- 172.21.15.104:0/609415611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35441a4090 msgr2=0x7f35441a4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35441a4090 0x7f35441a4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 -- 172.21.15.104:0/609415611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35441abe00 con 0x7f3530006230 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354a333700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f354419f280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3540010010 tx=0x7f3540012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3540011d20 con 0x7f3530006230 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354002a070 con 0x7f3530006230 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35441abfd0 con 0x7f3530006230 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.571+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354001d9a0 con 0x7f3530006230 2024-05-08T05:43:35.574 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.572+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35441ac3c0 con 0x7f3530006230 2024-05-08T05:43:35.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.572+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f354002c020 con 0x7f3530006230 2024-05-08T05:43:35.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.572+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3544064b00 con 0x7f3530006230 2024-05-08T05:43:35.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.573+0000 7f353b7fe700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f352803e9a0 0x7f3528040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:35.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.573+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f35400550f0 con 0x7f3530006230 2024-05-08T05:43:35.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.573+0000 7f3549b32700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f352803e9a0 0x7f3528040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:35.576 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.573+0000 7f3549b32700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f352803e9a0 0x7f3528040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3534009b80 tx=0x7f35340092c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:35.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.578+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3540003ba0 con 0x7f3530006230 2024-05-08T05:43:35.881 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.880+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f35441ac760 con 0x7f3530006230 2024-05-08T05:43:35.882 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.881+0000 7f353b7fe700 1 -- 172.21.15.104:0/609415611 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1277 (secure 0 0 0) 0x7f3540010ac0 con 0x7f3530006230 2024-05-08T05:43:35.882 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-05-08T05:43:35.882 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":3,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","modified":"2024-05-08T05:43:29.008504Z","created":"2024-05-08T05:41:30.871663Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:3300","nonce":0},{"type":"v1","addr":"172.21.15.3:6789","nonce":0}]},"addr":"172.21.15.3:6789/0","public_addr":"172.21.15.3:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:3300","nonce":0},{"type":"v1","addr":"172.21.15.89:6789","nonce":0}]},"addr":"172.21.15.89:6789/0","public_addr":"172.21.15.89:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-08T05:43:35.884 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f352803e9a0 msgr2=0x7f3528040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f352803e9a0 0x7f3528040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3534009b80 tx=0x7f35340092c0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 msgr2=0x7f354419f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f354419f280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3540010010 tx=0x7f3540012c00 comp rx=0 tx=0).stop 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 shutdown_connections 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35441a4090 0x7f35441a4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f352803e9a0 0x7f3528040e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3530006230 0x7f354419f280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 --2- 172.21.15.104:0/609415611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f354419f7c0 0x7f35441a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 >> 172.21.15.104:0/609415611 conn(0x7f35440fd9e0 msgr2=0x7f3544102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 shutdown_connections 2024-05-08T05:43:35.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:35.883+0000 7f354c597700 1 -- 172.21.15.104:0/609415611 wait complete. 2024-05-08T05:43:35.886 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 3 2024-05-08T05:43:36.518 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:36.518 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.518 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.518 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.518 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.518 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/609415611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.519 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:36.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:36.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:36.543 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T05:43:36.544 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:36.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/609415611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/609415611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.766 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.767 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.767 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.767 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:36.767 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:36.767 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:36.767 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:37.352 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:37.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:37 smithi089 ceph-mon[34495]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:37 smithi089 ceph-mon[34495]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:37 smithi089 ceph-mon[34495]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:37 smithi089 ceph-mon[34495]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:43:37.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:37 smithi089 ceph-mon[34495]: Reconfiguring daemon mon.a on smithi003 2024-05-08T05:43:37.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:37 smithi104 ceph-mon[34559]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:37 smithi104 ceph-mon[34559]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:37 smithi104 ceph-mon[34559]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:37 smithi104 ceph-mon[34559]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:43:37.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:37 smithi104 ceph-mon[34559]: Reconfiguring daemon mon.a on smithi003 2024-05-08T05:43:37.866 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:37 smithi003 ceph-mon[30137]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.866 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:37 smithi003 ceph-mon[30137]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.866 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:37 smithi003 ceph-mon[30137]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:43:37.866 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:37 smithi003 ceph-mon[30137]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:43:37.866 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:37 smithi003 ceph-mon[30137]: Reconfiguring daemon mon.a on smithi003 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:38.722 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:38.891 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 -- 172.21.15.3:0/2083850273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c108460 msgr2=0x7f3d6c108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 --2- 172.21.15.3:0/2083850273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c108460 0x7f3d6c108840 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b0a0 tx=0x7f3d6800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 -- 172.21.15.3:0/2083850273 shutdown_connections 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 --2- 172.21.15.3:0/2083850273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d6c103ef0 0x7f3d6c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 --2- 172.21.15.3:0/2083850273 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c108460 0x7f3d6c108840 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 --2- 172.21.15.3:0/2083850273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d6c103550 0x7f3d6c1039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 -- 172.21.15.3:0/2083850273 >> 172.21.15.3:0/2083850273 conn(0x7f3d6c07a600 msgr2=0x7f3d6c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 -- 172.21.15.3:0/2083850273 shutdown_connections 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.385+0000 7f3d73686700 1 -- 172.21.15.3:0/2083850273 wait complete. 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 Processor -- start 2024-05-08T05:43:39.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 -- start start 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 0x7f3d6c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d6c103ef0 0x7f3d6c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d6c108460 0x7f3d6c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1136e0 con 0x7f3d6c103550 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c113560 con 0x7f3d6c108460 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d73686700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c113860 con 0x7f3d6c103ef0 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d71422700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 0x7f3d6c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d71422700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 0x7f3d6c19f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42170/0 (socket says 172.21.15.3:42170) 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.386+0000 7f3d71422700 1 -- 172.21.15.3:0/768581059 learned_addr learned my addr 172.21.15.3:0/768581059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d70c21700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d6c103ef0 0x7f3d6c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71c23700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d6c108460 0x7f3d6c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71422700 1 -- 172.21.15.3:0/768581059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d6c103ef0 msgr2=0x7f3d6c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:39.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71422700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d6c103ef0 0x7f3d6c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71422700 1 -- 172.21.15.3:0/768581059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d6c108460 msgr2=0x7f3d6c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71422700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d6c108460 0x7f3d6c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71422700 1 -- 172.21.15.3:0/768581059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c19a160 con 0x7f3d6c103550 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d71422700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 0x7f3d6c19f840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b830 tx=0x7f3d68004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68021070 con 0x7f3d6c103550 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d68007e30 con 0x7f3d6c103550 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c19a3f0 con 0x7f3d6c103550 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680047d0 con 0x7f3d6c103550 2024-05-08T05:43:39.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.387+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c1a6710 con 0x7f3d6c103550 2024-05-08T05:43:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.388+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f3d68007980 con 0x7f3d6c103550 2024-05-08T05:43:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.388+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c10ba90 con 0x7f3d6c103550 2024-05-08T05:43:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.389+0000 7f3d627fc700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3d5803e750 0x7f3d58040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.389+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3d6800f020 con 0x7f3d6c103550 2024-05-08T05:43:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.389+0000 7f3d70c21700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3d5803e750 0x7f3d58040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:39.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.389+0000 7f3d70c21700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3d5803e750 0x7f3d58040c10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c07ad40 tx=0x7f3d5c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:39.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.394+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d6802a020 con 0x7f3d6c103550 2024-05-08T05:43:39.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:39 smithi089 ceph-mon[34495]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:43:39.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:39 smithi089 ceph-mon[34495]: Reconfiguring daemon mon.b on smithi089 2024-05-08T05:43:39.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.610+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f3d6c19af30 con 0x7f3d6c103550 2024-05-08T05:43:39.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.610+0000 7f3d627fc700 1 -- 172.21.15.3:0/768581059 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f3d680123f0 con 0x7f3d6c103550 2024-05-08T05:43:39.611 INFO:teuthology.orchestra.run.smithi003.stdout:# minimal ceph.conf for 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:39.611 INFO:teuthology.orchestra.run.smithi003.stdout:[global] 2024-05-08T05:43:39.611 INFO:teuthology.orchestra.run.smithi003.stdout: fsid = 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:43:39.611 INFO:teuthology.orchestra.run.smithi003.stdout: mon_host = [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-05-08T05:43:39.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3d5803e750 msgr2=0x7f3d58040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:39.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3d5803e750 0x7f3d58040c10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c07ad40 tx=0x7f3d5c0073d0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 msgr2=0x7f3d6c19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 0x7f3d6c19f840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800b830 tx=0x7f3d68004ce0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 shutdown_connections 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d6c108460 0x7f3d6c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3d5803e750 0x7f3d58040c10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3d6c103550 0x7f3d6c19f840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 --2- 172.21.15.3:0/768581059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d6c103ef0 0x7f3d6c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 >> 172.21.15.3:0/768581059 conn(0x7f3d6c07a600 msgr2=0x7f3d6c101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 shutdown_connections 2024-05-08T05:43:39.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:39.613+0000 7f3d73686700 1 -- 172.21.15.3:0/768581059 wait complete. 2024-05-08T05:43:39.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:39 smithi104 ceph-mon[34559]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:43:39.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:39 smithi104 ceph-mon[34559]: Reconfiguring daemon mon.b on smithi089 2024-05-08T05:43:39.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:39 smithi003 ceph-mon[30137]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:43:39.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:39 smithi003 ceph-mon[30137]: Reconfiguring daemon mon.b on smithi089 2024-05-08T05:43:40.219 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T05:43:40.220 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:43:40.220 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:43:40.251 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:43:40.252 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:40.319 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:43:40.319 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:43:40.350 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:43:40.351 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:40.417 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:43:40.417 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:43:40.443 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:43:40.443 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:40.507 INFO:tasks.cephadm:Adding mgr.x on smithi003 2024-05-08T05:43:40.507 INFO:tasks.cephadm:Adding mgr.y on smithi089 2024-05-08T05:43:40.507 INFO:tasks.cephadm:Adding mgr.z on smithi104 2024-05-08T05:43:40.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi003=x;smithi089=y;smithi104=z' 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: Reconfiguring daemon mon.c on smithi104 2024-05-08T05:43:40.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:40 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/768581059' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: Reconfiguring daemon mon.c on smithi104 2024-05-08T05:43:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:40 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/768581059' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: Reconfiguring daemon mon.c on smithi104 2024-05-08T05:43:40.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:40 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/768581059' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:41.206 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:43:42.348 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.348 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.348 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:42.348 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:42.348 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:42.349 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:42.349 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 -- 172.21.15.104:0/290004774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 msgr2=0x7f5850104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 --2- 172.21.15.104:0/290004774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850104b80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5840009990 tx=0x7f5840009ca0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.560 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 -- 172.21.15.104:0/290004774 shutdown_connections 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 --2- 172.21.15.104:0/290004774 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f58501050c0 0x7f585010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 --2- 172.21.15.104:0/290004774 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5850103520 0x7f5850103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 --2- 172.21.15.104:0/290004774 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850104b80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 -- 172.21.15.104:0/290004774 >> 172.21.15.104:0/290004774 conn(0x7f58500feac0 msgr2=0x7f5850100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 -- 172.21.15.104:0/290004774 shutdown_connections 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.557+0000 7f5857bcc700 1 -- 172.21.15.104:0/290004774 wait complete. 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 Processor -- start 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 -- start start 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5850103520 0x7f5850199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58501050c0 0x7f58501a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f585010f100 con 0x7f58501050c0 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f585010ef80 con 0x7f5850103520 2024-05-08T05:43:42.561 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5857bcc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f585010f280 con 0x7f5850104720 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:59130/0 (socket says 172.21.15.104:59130) 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 -- 172.21.15.104:0/2020272036 learned_addr learned my addr 172.21.15.104:0/2020272036 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855968700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5850103520 0x7f5850199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 -- 172.21.15.104:0/2020272036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5850103520 msgr2=0x7f5850199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5850103520 0x7f5850199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 -- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58501050c0 msgr2=0x7f58501a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5856169700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58501050c0 0x7f58501a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58501050c0 0x7f58501a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.558+0000 7f5855167700 1 -- 172.21.15.104:0/2020272036 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58501a13e0 con 0x7f5850104720 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.559+0000 7f5855167700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850199cc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f58400038a0 tx=0x7f58400057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.559+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840027070 con 0x7f5850104720 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.559+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5840004180 con 0x7f5850104720 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.559+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58501a1670 con 0x7f5850104720 2024-05-08T05:43:42.562 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.559+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584000f460 con 0x7f5850104720 2024-05-08T05:43:42.563 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.559+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58501a1b40 con 0x7f5850104720 2024-05-08T05:43:42.563 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.560+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7f584002a070 con 0x7f5850104720 2024-05-08T05:43:42.563 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.560+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585006a190 con 0x7f5850104720 2024-05-08T05:43:42.563 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.560+0000 7f5846ffd700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f583c03e750 0x7f583c040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:42.564 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.560+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f58400539a0 con 0x7f5850104720 2024-05-08T05:43:42.564 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.560+0000 7f5855968700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f583c03e750 0x7f583c040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:42.564 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.561+0000 7f5855968700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f583c03e750 0x7f583c040c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5850104580 tx=0x7f584c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:42.566 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.565+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5840013070 con 0x7f5850104720 2024-05-08T05:43:42.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.781+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi089=y;smithi104=z", "target": ["mon-mgr", ""]}) -- 0x7f58500638e0 con 0x7f583c03e750 2024-05-08T05:43:42.789 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.788+0000 7f5846ffd700 1 -- 172.21.15.104:0/2020272036 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f58500638e0 con 0x7f583c03e750 2024-05-08T05:43:42.789 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mgr update... 2024-05-08T05:43:42.791 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f583c03e750 msgr2=0x7f583c040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f583c03e750 0x7f583c040c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5850104580 tx=0x7f584c009190 comp rx=0 tx=0).stop 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 msgr2=0x7f5850199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850199cc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f58400038a0 tx=0x7f58400057c0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 shutdown_connections 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f583c03e750 0x7f583c040c10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f58501050c0 0x7f58501a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5850103520 0x7f5850199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 --2- 172.21.15.104:0/2020272036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5850104720 0x7f5850199cc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 >> 172.21.15.104:0/2020272036 conn(0x7f58500feac0 msgr2=0x7f5850107a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 shutdown_connections 2024-05-08T05:43:42.792 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:43:42.790+0000 7f5857bcc700 1 -- 172.21.15.104:0/2020272036 wait complete. 2024-05-08T05:43:43.340 DEBUG:teuthology.orchestra.run.smithi089:mgr.y> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T05:43:43.343 DEBUG:teuthology.orchestra.run.smithi104:mgr.z> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T05:43:43.345 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T05:43:43.345 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:43:43.346 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:43:43.366 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:43:43.366 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_1 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-08 05:42:46.200366222 +0000 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-08 05:37:34.886950481 +0000 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-08 05:37:34.886950481 +0000 2024-05-08T05:43:43.425 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-08 05:37:34.886950481 +0000 2024-05-08T05:43:43.426 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:43:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-08T05:43:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-08T05:43:43.494 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000225105 s, 2.3 MB/s 2024-05-08T05:43:43.496 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:43:43.555 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_2 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-08 05:42:46.228365925 +0000 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-08 05:37:35.257946542 +0000 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-08 05:37:35.257946542 +0000 2024-05-08T05:43:43.615 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-08 05:37:35.257946542 +0000 2024-05-08T05:43:43.616 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:43:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-08T05:43:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-08T05:43:43.684 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000197309 s, 2.6 MB/s 2024-05-08T05:43:43.685 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:43:43.746 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_3 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-08 05:42:46.260365585 +0000 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-08 05:37:35.604942858 +0000 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-08 05:37:35.604942858 +0000 2024-05-08T05:43:43.805 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-08 05:37:35.604942858 +0000 2024-05-08T05:43:43.806 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:43:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-08T05:43:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-08T05:43:43.873 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000286783 s, 1.8 MB/s 2024-05-08T05:43:43.874 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:43:43.933 DEBUG:teuthology.orchestra.run.smithi003:> stat /dev/vg_nvme/lv_4 2024-05-08T05:43:43.991 INFO:teuthology.orchestra.run.smithi003.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:43:43.991 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:43.991 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-08T05:43:43.991 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:43.992 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:43.992 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-08 05:42:46.295365214 +0000 2024-05-08T05:43:43.992 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-08 05:37:35.953939152 +0000 2024-05-08T05:43:43.992 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-08 05:37:35.953939152 +0000 2024-05-08T05:43:43.992 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-08 05:37:35.953939152 +0000 2024-05-08T05:43:43.992 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:43:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi089=y;smithi104=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: Saving service mgr spec with placement smithi003=x;smithi089=y;smithi104=z;count:3 2024-05-08T05:43:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: Deploying daemon mgr.z on smithi104 2024-05-08T05:43:44.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:43 smithi104 ceph-mon[34559]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi089=y;smithi104=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: Saving service mgr spec with placement smithi003=x;smithi089=y;smithi104=z;count:3 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:43:44.055 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:44.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:44.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: Deploying daemon mgr.z on smithi104 2024-05-08T05:43:44.056 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:43 smithi003 ceph-mon[30137]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records in 2024-05-08T05:43:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:1+0 records out 2024-05-08T05:43:44.057 INFO:teuthology.orchestra.run.smithi003.stderr:512 bytes copied, 0.000207345 s, 2.5 MB/s 2024-05-08T05:43:44.059 DEBUG:teuthology.orchestra.run.smithi003:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:43:44.117 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-05-08T05:43:44.117 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:43:44.136 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:43:44.136 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-05-08T05:43:44.193 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:43:44.193 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-05-08 05:43:28.497815782 +0000 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-05-08 05:37:36.477122311 +0000 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-05-08 05:37:36.477122311 +0000 2024-05-08T05:43:44.194 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-05-08 05:37:36.477122311 +0000 2024-05-08T05:43:44.194 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi003=x;smithi089=y;smithi104=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: Saving service mgr spec with placement smithi003=x;smithi089=y;smithi104=z;count:3 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:44.257 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:44.258 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:43:44.258 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:44.258 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:44.258 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: Deploying daemon mgr.z on smithi104 2024-05-08T05:43:44.258 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:43 smithi089 ceph-mon[34495]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-05-08T05:43:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-05-08T05:43:44.259 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000221263 s, 2.3 MB/s 2024-05-08T05:43:44.261 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:43:44.319 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-05-08T05:43:44.375 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-05-08 05:43:28.497815782 +0000 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-05-08 05:37:36.838114760 +0000 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-05-08 05:37:36.838114760 +0000 2024-05-08T05:43:44.376 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-05-08 05:37:36.838114760 +0000 2024-05-08T05:43:44.376 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:43:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-05-08T05:43:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-05-08T05:43:44.440 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000246714 s, 2.1 MB/s 2024-05-08T05:43:44.442 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:43:44.499 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-05-08 05:43:28.498815762 +0000 2024-05-08T05:43:44.555 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-05-08 05:37:37.157108087 +0000 2024-05-08T05:43:44.556 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-05-08 05:37:37.157108087 +0000 2024-05-08T05:43:44.556 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-05-08 05:37:37.157108087 +0000 2024-05-08T05:43:44.556 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-05-08T05:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-05-08T05:43:44.618 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000204975 s, 2.5 MB/s 2024-05-08T05:43:44.619 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:43:44.676 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:44.731 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-05-08 05:43:28.498815762 +0000 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-05-08 05:37:37.488101163 +0000 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-05-08 05:37:37.488101163 +0000 2024-05-08T05:43:44.732 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-05-08 05:37:37.488101163 +0000 2024-05-08T05:43:44.732 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:43:44.794 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-05-08T05:43:44.794 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-05-08T05:43:44.794 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000174766 s, 2.9 MB/s 2024-05-08T05:43:44.795 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:43:44.851 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-05-08T05:43:44.851 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:43:44.867 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:43:44.867 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-05-08T05:43:44.922 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:43:44.922 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:44.922 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T05:43:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-08 05:43:44.745899731 +0000 2024-05-08T05:43:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-08 05:37:40.286362136 +0000 2024-05-08T05:43:44.923 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-08 05:37:40.286362136 +0000 2024-05-08T05:43:44.923 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-08 05:37:40.286362136 +0000 2024-05-08T05:43:44.923 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:43:44.987 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-08T05:43:44.987 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-08T05:43:44.988 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000205431 s, 2.5 MB/s 2024-05-08T05:43:44.989 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:43:45.044 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-05-08T05:43:45.099 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-08 05:43:44.746899720 +0000 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-08 05:37:40.640357993 +0000 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-08 05:37:40.640357993 +0000 2024-05-08T05:43:45.100 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-08 05:37:40.640357993 +0000 2024-05-08T05:43:45.100 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:43:45.164 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-08T05:43:45.164 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-08T05:43:45.165 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000160391 s, 3.2 MB/s 2024-05-08T05:43:45.166 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:43:45.221 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-05-08T05:43:45.275 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:43:45.275 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:45.275 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T05:43:45.275 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:45.275 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:45.276 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-08 05:43:44.746899720 +0000 2024-05-08T05:43:45.276 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-08 05:37:40.963354213 +0000 2024-05-08T05:43:45.276 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-08 05:37:40.963354213 +0000 2024-05-08T05:43:45.276 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-08 05:37:40.963354213 +0000 2024-05-08T05:43:45.276 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:43:45.338 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-08T05:43:45.338 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-08T05:43:45.338 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000167299 s, 3.1 MB/s 2024-05-08T05:43:45.340 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:43:45.395 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-05-08 05:43:44.746899720 +0000 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-05-08 05:37:41.306350199 +0000 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-05-08 05:37:41.306350199 +0000 2024-05-08T05:43:45.451 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-05-08 05:37:41.306350199 +0000 2024-05-08T05:43:45.452 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:43:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-05-08T05:43:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-05-08T05:43:45.517 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000174403 s, 2.9 MB/s 2024-05-08T05:43:45.518 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:43:45.575 INFO:tasks.cephadm:Deploying osd.0 on smithi003 with /dev/vg_nvme/lv_4... 2024-05-08T05:43:45.575 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:43:45.734 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:45.542+0000 7f374c946200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:43:45.735 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:45.733+0000 7f374c946200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:43:45.830 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:46.021 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:45.837+0000 7f374c946200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:43:46.521 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:46.150+0000 7f374c946200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:43:46.521 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:46.234+0000 7f374c946200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:43:46.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:46 smithi089 ceph-mon[34495]: Deploying daemon mgr.y on smithi089 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:46.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:46 smithi003 ceph-mon[30137]: Deploying daemon mgr.y on smithi089 2024-05-08T05:43:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:43:47.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:47.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:47.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:46 smithi104 ceph-mon[34559]: Deploying daemon mgr.y on smithi089 2024-05-08T05:43:47.750 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:43:47.750 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:43:47.750 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-05-08T05:43:47.750 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-05-08T05:43:47.750 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121069 s, 866 MB/s 2024-05-08T05:43:47.750 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-05-08T05:43:47.759 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:47.421+0000 7f374c946200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:43:47.759 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:47.591+0000 7f374c946200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:43:47.759 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:47.758+0000 7f374c946200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:43:47.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:47 smithi089 podman[35902]: 2024-05-08 05:43:47.586988577 +0000 UTC m=+0.017744345 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:43:48.029 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:47 smithi089 podman[35902]: 2024-05-08 05:43:47.874718262 +0000 UTC m=+0.305474016 container init bf127d7b51728de569e5cda7cc830d07a275c1bb8edd20fceb019d7b6db8070f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-08T05:43:48.029 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:47 smithi089 podman[35902]: 2024-05-08 05:43:47.877120483 +0000 UTC m=+0.307876228 container start bf127d7b51728de569e5cda7cc830d07a275c1bb8edd20fceb019d7b6db8070f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:43:48.029 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:47 smithi089 ceph-mgr[35921]: -- 172.21.15.89:0/2893172388 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56208adb25a0 con 0x56208ad86c00 2024-05-08T05:43:48.029 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:47 smithi089 bash[35902]: bf127d7b51728de569e5cda7cc830d07a275c1bb8edd20fceb019d7b6db8070f 2024-05-08T05:43:48.029 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:47 smithi089 systemd[1]: Started Ceph mgr.y for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:43:48.321 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:48.028+0000 7fc569010200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:43:48.321 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:48.213+0000 7fc569010200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:43:48.321 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:48.320+0000 7fc569010200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:43:48.343 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:48.044+0000 7f374c946200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:43:48.344 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:48.136+0000 7f374c946200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:43:48.344 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:48.343+0000 7f374c946200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:43:48.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_4 2024-05-08T05:43:48.633 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.644 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:48.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:48.539+0000 7f374c946200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:49.012 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:48.643+0000 7fc569010200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:43:49.013 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:48.749+0000 7fc569010200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:43:49.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:49.172+0000 7f374c946200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:43:49.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:49.390+0000 7f374c946200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:43:50.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 -- 172.21.15.3:0/1026165046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358103fa0 msgr2=0x7fe35810a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 --2- 172.21.15.3:0/1026165046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358103fa0 0x7fe35810a850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe35000b0d0 tx=0x7fe35000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 -- 172.21.15.3:0/1026165046 shutdown_connections 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 --2- 172.21.15.3:0/1026165046 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358103fa0 0x7fe35810a850 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 --2- 172.21.15.3:0/1026165046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358102400 0x7fe358102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 --2- 172.21.15.3:0/1026165046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 0x7fe358103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 -- 172.21.15.3:0/1026165046 >> 172.21.15.3:0/1026165046 conn(0x7fe3580fd9a0 msgr2=0x7fe3580ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 -- 172.21.15.3:0/1026165046 shutdown_connections 2024-05-08T05:43:50.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.200+0000 7fe35ec25700 1 -- 172.21.15.3:0/1026165046 wait complete. 2024-05-08T05:43:50.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 Processor -- start 2024-05-08T05:43:50.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 -- start start 2024-05-08T05:43:50.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358102400 0x7fe35806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 0x7fe35819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358103fa0 0x7fe3581a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe35810de80 con 0x7fe358102400 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35c9c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358102400 0x7fe35806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe35810dd00 con 0x7fe358103600 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35ec25700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe35810e000 con 0x7fe358103fa0 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35c9c1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358102400 0x7fe35806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34442/0 (socket says 172.21.15.3:34442) 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35c9c1700 1 -- 172.21.15.3:0/3595916618 learned_addr learned my addr 172.21.15.3:0/3595916618 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe357fff700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 0x7fe35819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.201+0000 7fe35d1c2700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358103fa0 0x7fe3581a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35d1c2700 1 -- 172.21.15.3:0/3595916618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 msgr2=0x7fe35819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35d1c2700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 0x7fe35819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35d1c2700 1 -- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358102400 msgr2=0x7fe35806bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35d1c2700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358102400 0x7fe35806bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35d1c2700 1 -- 172.21.15.3:0/3595916618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3581a1280 con 0x7fe358103fa0 2024-05-08T05:43:50.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe357fff700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 0x7fe35819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:43:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35d1c2700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358103fa0 0x7fe3581a09b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe35000d010 tx=0x7fe3500079a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe350003a00 con 0x7fe358103fa0 2024-05-08T05:43:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe350003ba0 con 0x7fe358103fa0 2024-05-08T05:43:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3581a1510 con 0x7fe358103fa0 2024-05-08T05:43:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.202+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35001b7d0 con 0x7fe358103fa0 2024-05-08T05:43:50.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.203+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3581a1910 con 0x7fe358103fa0 2024-05-08T05:43:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.203+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 13) ==== 50277+0+0 (secure 0 0 0) 0x7fe35001b970 con 0x7fe358103fa0 2024-05-08T05:43:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.203+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe35802e510 con 0x7fe358103fa0 2024-05-08T05:43:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.204+0000 7fe355ffb700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe34803e750 0x7fe348040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:43:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.204+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe350053fc0 con 0x7fe358103fa0 2024-05-08T05:43:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.204+0000 7fe35c9c1700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe34803e750 0x7fe348040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:43:50.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.204+0000 7fe35c9c1700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe34803e750 0x7fe348040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe34c007cf0 tx=0x7fe34c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:43:50.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.209+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe350024350 con 0x7fe358103fa0 2024-05-08T05:43:50.262 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:49 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:49.988+0000 7fc569010200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:43:50.262 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:50.156+0000 7fc569010200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:43:50.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:49.819+0000 7f374c946200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:43:50.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:49.913+0000 7f374c946200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:43:50.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:50.438+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe3580638e0 con 0x7fe34803e750 2024-05-08T05:43:50.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:50.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: Reconfiguring daemon mgr.x on smithi003 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:43:50.543 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.613 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:50.326+0000 7fc569010200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:43:50.613 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:50.613+0000 7fc569010200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.614 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: Reconfiguring daemon mgr.x on smithi003 2024-05-08T05:43:50.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:43:50.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:43:50.615 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.626 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:50.275+0000 7f374c946200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:43:50.626 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:50.626+0000 7f374c946200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:43:50.626 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: Reconfiguring daemon mgr.x on smithi003 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:43:50.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:50.906 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:50.704+0000 7fc569010200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:43:50.906 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:50.906+0000 7fc569010200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:43:50.956 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:50.718+0000 7f374c946200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:43:51.262 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:51.097+0000 7fc569010200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:43:51.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:50.955+0000 7f374c946200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:43:51.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:51.086+0000 7f374c946200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:43:51.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:51.180+0000 7f374c946200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='client.24130 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:51.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.762 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:51.699+0000 7fc569010200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='client.24130 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:51.504+0000 7f374c946200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:51.594+0000 7f374c946200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='client.24130 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:43:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:43:52.263 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:51.916+0000 7fc569010200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:43:52.457 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:52.186+0000 7f374c946200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:43:52.457 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:43:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:43:52.397+0000 7f374c946200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:43:52.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:52.467+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 14) ==== 99936+0+0 (secure 0 0 0) 0x7fe35001d970 con 0x7fe358103fa0 2024-05-08T05:43:52.565 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:52 smithi003 ceph-mon[30137]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:52.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:52 smithi003 ceph-mon[30137]: Standby manager daemon z started 2024-05-08T05:43:52.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:52 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:43:52.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:52 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:43:52.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:52 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:43:52.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:52 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:52.331+0000 7fc569010200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:52.426+0000 7fc569010200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:52 smithi089 ceph-mon[34495]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:52 smithi089 ceph-mon[34495]: Standby manager daemon z started 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:52 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:52 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:52 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:43:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:52 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:43:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:52 smithi104 ceph-mon[34559]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:52 smithi104 ceph-mon[34559]: Standby manager daemon z started 2024-05-08T05:43:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:52 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:43:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:52 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:43:52.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:52 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:43:52.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:52 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3015530816' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:43:53.142 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:52.792+0000 7fc569010200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:43:53.142 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:53.141+0000 7fc569010200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:43:53.471 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:53.233+0000 7fc569010200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:53.474+0000 7fc569010200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:53.605+0000 7fc569010200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:53.698+0000 7fc569010200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:53 smithi089 ceph-mon[34495]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:53 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3528005265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a64e1a5-0f5b-4a16-b521-3b41c78d7269"}]: dispatch 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:53 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3528005265' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a64e1a5-0f5b-4a16-b521-3b41c78d7269"}]': finished 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:53 smithi089 ceph-mon[34495]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:43:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:43:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:53 smithi104 ceph-mon[34559]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-08T05:43:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:43:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:53 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3528005265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a64e1a5-0f5b-4a16-b521-3b41c78d7269"}]: dispatch 2024-05-08T05:43:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:53 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3528005265' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a64e1a5-0f5b-4a16-b521-3b41c78d7269"}]': finished 2024-05-08T05:43:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:53 smithi104 ceph-mon[34559]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:43:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:43:53.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:53 smithi003 ceph-mon[30137]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-08T05:43:53.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:43:53.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:53 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3528005265' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a64e1a5-0f5b-4a16-b521-3b41c78d7269"}]: dispatch 2024-05-08T05:43:53.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:53 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3528005265' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a64e1a5-0f5b-4a16-b521-3b41c78d7269"}]': finished 2024-05-08T05:43:53.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:53 smithi003 ceph-mon[30137]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:43:53.804 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:43:54.473 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:54.019+0000 7fc569010200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:43:54.473 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:54.108+0000 7fc569010200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:43:54.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:54 smithi089 ceph-mon[34495]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:54.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:54 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1929425568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:43:54.763 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:54.703+0000 7fc569010200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:43:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:54 smithi104 ceph-mon[34559]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:54 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1929425568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:43:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:54 smithi003 ceph-mon[30137]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:54 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1929425568' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:43:55.262 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:43:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:43:54.915+0000 7fc569010200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:43:55.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:43:55.486+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mon.1 v2:172.21.15.104:3300/0 8 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fe35001d450 con 0x7fe358103fa0 2024-05-08T05:43:55.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:55 smithi089 ceph-mon[34495]: Standby manager daemon y started 2024-05-08T05:43:55.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:55 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:43:55.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:55 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:43:55.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:55 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:43:55.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:55 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:43:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:55 smithi104 ceph-mon[34559]: Standby manager daemon y started 2024-05-08T05:43:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:55 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:43:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:55 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:43:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:55 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:43:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:55 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:43:55.811 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:55 smithi003 ceph-mon[30137]: Standby manager daemon y started 2024-05-08T05:43:55.811 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:55 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:43:55.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:55 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:43:55.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:55 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:43:55.812 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:55 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2290149258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:43:56.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:56 smithi089 ceph-mon[34495]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:56 smithi089 ceph-mon[34495]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-05-08T05:43:56.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:43:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:56 smithi104 ceph-mon[34559]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:56 smithi104 ceph-mon[34559]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-05-08T05:43:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:43:56.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:56 smithi003 ceph-mon[30137]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:56.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:56 smithi003 ceph-mon[30137]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-05-08T05:43:56.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:43:58.758 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:43:58 smithi003 ceph-mon[30137]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:58.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:43:58 smithi089 ceph-mon[34495]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:43:58.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:43:58 smithi104 ceph-mon[34559]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:00.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:00 smithi089 ceph-mon[34495]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:00.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:00 smithi104 ceph-mon[34559]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:00.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:00 smithi003 ceph-mon[30137]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:01.628 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:44:01.629 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:01.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:44:01.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:01.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:44:01.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:02.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:02 smithi003 ceph-mon[30137]: Deploying daemon osd.0 on smithi003 2024-05-08T05:44:02.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:02 smithi003 ceph-mon[30137]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:02 smithi089 ceph-mon[34495]: Deploying daemon osd.0 on smithi003 2024-05-08T05:44:02.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:02 smithi089 ceph-mon[34495]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:02.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:02 smithi104 ceph-mon[34559]: Deploying daemon osd.0 on smithi003 2024-05-08T05:44:02.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:02 smithi104 ceph-mon[34559]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:04 smithi003 ceph-mon[30137]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:04.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:04 smithi089 ceph-mon[34495]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:04.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:04 smithi104 ceph-mon[34559]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:06.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:06 smithi089 ceph-mon[34495]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:06.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:06 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:06.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:06 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:06.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:06 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:06 smithi104 ceph-mon[34559]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:06 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:06 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:06 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:06 smithi003 ceph-mon[30137]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:08.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:08 smithi089 ceph-mon[34495]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:08.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:08 smithi104 ceph-mon[34559]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:08.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:08 smithi003 ceph-mon[30137]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:09.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.662+0000 7fe355ffb700 1 -- 172.21.15.3:0/3595916618 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe3580638e0 con 0x7fe34803e750 2024-05-08T05:44:09.663 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 0 on host 'smithi003' 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe34803e750 msgr2=0x7fe348040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe34803e750 0x7fe348040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe34c007cf0 tx=0x7fe34c007480 comp rx=0 tx=0).stop 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358103fa0 msgr2=0x7fe3581a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358103fa0 0x7fe3581a09b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe35000d010 tx=0x7fe3500079a0 comp rx=0 tx=0).stop 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 shutdown_connections 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe358103fa0 0x7fe3581a09b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:09.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe34803e750 0x7fe348040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:09.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe358102400 0x7fe35806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:09.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 --2- 172.21.15.3:0/3595916618 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe358103600 0x7fe35819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:09.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.664+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 >> 172.21.15.3:0/3595916618 conn(0x7fe3580fd9a0 msgr2=0x7fe3581069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:09.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.665+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 shutdown_connections 2024-05-08T05:44:09.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:44:09.665+0000 7fe35ec25700 1 -- 172.21.15.3:0/3595916618 wait complete. 2024-05-08T05:44:10.253 DEBUG:teuthology.orchestra.run.smithi003:osd.0> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T05:44:10.255 INFO:tasks.cephadm:Deploying osd.1 on smithi089 with /dev/vg_nvme/lv_4... 2024-05-08T05:44:10.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:44:10.498 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:44:10.854 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:10 smithi089 ceph-mon[34495]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:10.854 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.854 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:10 smithi089 ceph-mon[34495]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:10.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:10 smithi003 ceph-mon[30137]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:10 smithi003 ceph-mon[30137]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:10 smithi104 ceph-mon[34559]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:10 smithi104 ceph-mon[34559]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:44:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:11.709 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:11 smithi089 ceph-mon[34495]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:44:11.710 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:11 smithi089 ceph-mon[34495]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:44:11.710 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:11 smithi089 ceph-mon[34495]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:44:11.710 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:11 smithi003 ceph-mon[30137]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:44:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:11 smithi003 ceph-mon[30137]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:44:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:11 smithi003 ceph-mon[30137]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:44:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:11 smithi104 ceph-mon[34559]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:44:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:11 smithi104 ceph-mon[34559]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:44:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:11 smithi104 ceph-mon[34559]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:44:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:44:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:44:12.269 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2024-05-08T05:44:12.269 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2024-05-08T05:44:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147959 s, 709 MB/s 2024-05-08T05:44:12.270 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2024-05-08T05:44:12.337 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:44:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0[44525]: 2024-05-08T05:44:12.082+0000 7fa429b1d700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T05:44:12.338 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:44:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0[44525]: 2024-05-08T05:44:12.085+0000 7fa421908700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:44:12.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:12 smithi089 ceph-mon[34495]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:12.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:12 smithi089 ceph-mon[34495]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:44:12.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:12 smithi089 ceph-mon[34495]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:44:12.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:12 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:12.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:12 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:12.832 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:12 smithi089 ceph-mon[34495]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' 2024-05-08T05:44:12.860 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_4 2024-05-08T05:44:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:12 smithi003 ceph-mon[30137]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:12 smithi003 ceph-mon[30137]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:44:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:12 smithi003 ceph-mon[30137]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:44:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:12 smithi003 ceph-mon[30137]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' 2024-05-08T05:44:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:12 smithi104 ceph-mon[34559]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:12 smithi104 ceph-mon[34559]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:44:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:12 smithi104 ceph-mon[34559]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:44:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:12 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:12 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:12 smithi104 ceph-mon[34559]: from='osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717]' entity='osd.0' 2024-05-08T05:44:13.101 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:44:13.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:13 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:44:13.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:13 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:44:13.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:13 smithi089 ceph-mon[34495]: osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] boot 2024-05-08T05:44:13.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:13 smithi089 ceph-mon[34495]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:44:13.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:13 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:44:13.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:13 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:44:13.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.844 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:13 smithi003 ceph-mon[30137]: osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] boot 2024-05-08T05:44:13.844 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:13 smithi003 ceph-mon[30137]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:44:13.844 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.922 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:13 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:44:13.922 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:13 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:44:13.923 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:13.923 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:13 smithi104 ceph-mon[34559]: osd.0 [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] boot 2024-05-08T05:44:13.923 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:13 smithi104 ceph-mon[34559]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:44:13.923 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:44:14.545 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 -- 172.21.15.89:0/1706141203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b441091f0 msgr2=0x7f2b4410b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/1706141203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b441091f0 0x7f2b4410b660 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c00b0a0 tx=0x7f2b3c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 -- 172.21.15.89:0/1706141203 shutdown_connections 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/1706141203 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b441091f0 0x7f2b4410b660 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/1706141203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b44103f90 0x7f2b44106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/1706141203 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 0x7f2b44108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 -- 172.21.15.89:0/1706141203 >> 172.21.15.89:0/1706141203 conn(0x7f2b440fd9a0 msgr2=0x7f2b440ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 -- 172.21.15.89:0/1706141203 shutdown_connections 2024-05-08T05:44:14.546 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.545+0000 7f2b4a80f700 1 -- 172.21.15.89:0/1706141203 wait complete. 2024-05-08T05:44:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 Processor -- start 2024-05-08T05:44:14.548 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 -- start start 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 0x7f2b4419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 0x7f2b4419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b441091f0 0x7f2b441a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410efc0 con 0x7f2b441068c0 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410ee40 con 0x7f2b44103f90 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b4a80f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4410f140 con 0x7f2b441091f0 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b43fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 0x7f2b4419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b43fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 0x7f2b4419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36724/0 (socket says 172.21.15.89:36724) 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b43fff700 1 -- 172.21.15.89:0/3808657500 learned_addr learned my addr 172.21.15.89:0/3808657500 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.546+0000 7f2b48dac700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b441091f0 0x7f2b441a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b437fe700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 0x7f2b4419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b43fff700 1 -- 172.21.15.89:0/3808657500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b441091f0 msgr2=0x7f2b441a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b43fff700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b441091f0 0x7f2b441a2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.549 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b43fff700 1 -- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 msgr2=0x7f2b4419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b43fff700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 0x7f2b4419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b43fff700 1 -- 172.21.15.89:0/3808657500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b441a3480 con 0x7f2b44103f90 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b437fe700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 0x7f2b4419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b48dac700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b441091f0 0x7f2b441a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b43fff700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 0x7f2b4419b820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400e9c0 tx=0x7f2b3400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3400cbd0 con 0x7f2b44103f90 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b3400cd70 con 0x7f2b44103f90 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b441a3770 con 0x7f2b44103f90 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b34005680 con 0x7f2b44103f90 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.547+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b441a3c60 con 0x7f2b44103f90 2024-05-08T05:44:14.550 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.548+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4406a190 con 0x7f2b44103f90 2024-05-08T05:44:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.549+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f2b34004750 con 0x7f2b44103f90 2024-05-08T05:44:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.550+0000 7f2b417fa700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2b2c0b27f0 0x7f2b2c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.550+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f2b340e0cc0 con 0x7f2b44103f90 2024-05-08T05:44:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.550+0000 7f2b437fe700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2b2c0b27f0 0x7f2b2c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:14.551 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.551+0000 7f2b437fe700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2b2c0b27f0 0x7f2b2c0b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009cb0 tx=0x7f2b38009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:14.554 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.554+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2b34054df0 con 0x7f2b44103f90 2024-05-08T05:44:14.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:14.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:14 smithi089 ceph-mon[34495]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:44:14.766 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:14.766+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2b440638e0 con 0x7f2b2c0b27f0 2024-05-08T05:44:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:14.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:14.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:14.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:14.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:14 smithi003 ceph-mon[30137]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:15.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:14 smithi104 ceph-mon[34559]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:44:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi003 2024-05-08T05:44:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi003 to 17093M 2024-05-08T05:44:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:15.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:15.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:15.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi003 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi003 to 17093M 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi003 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi003 to 17093M 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:16.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:16.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:16 smithi089 ceph-mon[34495]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:16.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:16 smithi089 ceph-mon[34495]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:16 smithi003 ceph-mon[30137]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:16 smithi003 ceph-mon[30137]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:16 smithi104 ceph-mon[34559]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:16 smithi104 ceph-mon[34559]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:17 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/3094162006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]: dispatch 2024-05-08T05:44:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:17 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]: dispatch 2024-05-08T05:44:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:17 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]': finished 2024-05-08T05:44:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:17 smithi003 ceph-mon[30137]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:44:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:17 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:17 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/3094162006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]: dispatch 2024-05-08T05:44:18.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:17 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]: dispatch 2024-05-08T05:44:18.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:17 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]': finished 2024-05-08T05:44:18.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:17 smithi089 ceph-mon[34495]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:44:18.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:17 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:17 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/3094162006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]: dispatch 2024-05-08T05:44:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:17 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]: dispatch 2024-05-08T05:44:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:17 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "804db558-09e6-4d7d-9eca-b70cd45c71a8"}]': finished 2024-05-08T05:44:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:17 smithi104 ceph-mon[34559]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:44:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:17 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:18 smithi003 ceph-mon[30137]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:18 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/3257128072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:19.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:18 smithi089 ceph-mon[34495]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:19.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:18 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/3257128072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:18 smithi104 ceph-mon[34559]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:18 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/3257128072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:20 smithi003 ceph-mon[30137]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:21.009 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:20 smithi089 ceph-mon[34495]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:20 smithi104 ceph-mon[34559]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:22 smithi003 ceph-mon[30137]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:23.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:22 smithi089 ceph-mon[34495]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:23.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:22 smithi104 ceph-mon[34559]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:24.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:24 smithi089 ceph-mon[34495]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:24.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:24 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:44:24.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:24 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:24 smithi003 ceph-mon[30137]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:24 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:44:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:24 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:24 smithi104 ceph-mon[34559]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:24 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:44:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:24 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:25.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:25 smithi089 ceph-mon[34495]: Deploying daemon osd.1 on smithi089 2024-05-08T05:44:25.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:25 smithi003 ceph-mon[30137]: Deploying daemon osd.1 on smithi089 2024-05-08T05:44:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:25 smithi104 ceph-mon[34559]: Deploying daemon osd.1 on smithi089 2024-05-08T05:44:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:26 smithi003 ceph-mon[30137]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:27.002 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:26 smithi089 ceph-mon[34495]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:26 smithi104 ceph-mon[34559]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:28 smithi003 ceph-mon[30137]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:28.952 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:28 smithi089 ceph-mon[34495]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:28 smithi104 ceph-mon[34559]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:30.711 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:30 smithi089 ceph-mon[34495]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:30.711 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:30.712 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:30.712 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:30 smithi104 ceph-mon[34559]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:31.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:30 smithi003 ceph-mon[30137]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:31.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:31.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:31.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:32.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:32 smithi089 ceph-mon[34495]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:32 smithi104 ceph-mon[34559]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:32 smithi003 ceph-mon[30137]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:33.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:33.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:33.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:33.433 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.432+0000 7f2b417fa700 1 -- 172.21.15.89:0/3808657500 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2b440638e0 con 0x7f2b2c0b27f0 2024-05-08T05:44:33.433 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 1 on host 'smithi089' 2024-05-08T05:44:33.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2b2c0b27f0 msgr2=0x7f2b2c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:33.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2b2c0b27f0 0x7f2b2c0b4cb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009cb0 tx=0x7f2b38009260 comp rx=0 tx=0).stop 2024-05-08T05:44:33.435 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 msgr2=0x7f2b4419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 0x7f2b4419b820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b3400e9c0 tx=0x7f2b3400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 shutdown_connections 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b441091f0 0x7f2b441a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b44103f90 0x7f2b4419b820 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2b2c0b27f0 0x7f2b2c0b4cb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 --2- 172.21.15.89:0/3808657500 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2b441068c0 0x7f2b4419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 >> 172.21.15.89:0/3808657500 conn(0x7f2b440fd9a0 msgr2=0x7f2b441036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 shutdown_connections 2024-05-08T05:44:33.436 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:44:33.435+0000 7f2b4a80f700 1 -- 172.21.15.89:0/3808657500 wait complete. 2024-05-08T05:44:34.024 DEBUG:teuthology.orchestra.run.smithi089:osd.1> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T05:44:34.026 INFO:tasks.cephadm:Deploying osd.2 on smithi104 with /dev/vg_nvme/lv_4... 2024-05-08T05:44:34.026 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:44:34.272 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:44:34.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:34 smithi003 ceph-mon[30137]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:34.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:34.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:34.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:34.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:34 smithi003 ceph-mon[30137]: from='osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:44:34.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:34 smithi003 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:44:34.710 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:34 smithi104 ceph-mon[34559]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:34.710 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:34.710 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:34.710 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:34.710 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:34 smithi104 ceph-mon[34559]: from='osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:44:34.710 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:34 smithi104 ceph-mon[34559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:44:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:34 smithi089 ceph-mon[34495]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:34 smithi089 ceph-mon[34495]: from='osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:44:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:34 smithi089 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:44:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:35 smithi089 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:44:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:35 smithi089 ceph-mon[34495]: from='osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:44:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:35 smithi089 ceph-mon[34495]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:44:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:35 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:35 smithi089 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:44:35.763 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:44:35 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-08T05:44:35.676+0000 7f34946cc700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T05:44:35.764 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:44:35 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-08T05:44:35.681+0000 7f34904bf700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:44:36.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:35 smithi104 ceph-mon[34559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:44:36.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:35 smithi104 ceph-mon[34559]: from='osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:44:36.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:35 smithi104 ceph-mon[34559]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:44:36.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:35 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:36.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:35 smithi104 ceph-mon[34559]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:44:36.109 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:44:36.109 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:44:36.109 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-08T05:44:36.109 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-08T05:44:36.109 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125341 s, 837 MB/s 2024-05-08T05:44:36.109 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-08T05:44:36.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:35 smithi003 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:44:36.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:35 smithi003 ceph-mon[30137]: from='osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:44:36.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:35 smithi003 ceph-mon[30137]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:44:36.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:35 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:36.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:35 smithi003 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:44:36.707 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-05-08T05:44:36.859 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:36 smithi089 ceph-mon[34495]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:36.859 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:36 smithi089 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:44:36.859 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:36 smithi089 ceph-mon[34495]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:44:36.859 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:36.859 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:36.949 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:44:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:36 smithi104 ceph-mon[34559]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:36 smithi104 ceph-mon[34559]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:44:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:36 smithi104 ceph-mon[34559]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:44:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:37.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:36 smithi003 ceph-mon[30137]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:37.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:36 smithi003 ceph-mon[30137]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:44:37.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:36 smithi003 ceph-mon[30137]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:44:37.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:37.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:37.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:44:37.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:44:37.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] boot 2024-05-08T05:44:37.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:44:37.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:37.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] boot 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:38.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:38.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: osd.1 [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] boot 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:44:38.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:38.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:38.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.512+0000 7f3bd1808700 1 -- 172.21.15.104:0/1476930117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc103540 msgr2=0x7f3bcc103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.512+0000 7f3bd1808700 1 --2- 172.21.15.104:0/1476930117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc103540 0x7f3bcc103960 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3bc400b0a0 tx=0x7f3bc400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 -- 172.21.15.104:0/1476930117 shutdown_connections 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 --2- 172.21.15.104:0/1476930117 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc10b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 --2- 172.21.15.104:0/1476930117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc103540 0x7f3bcc103960 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 --2- 172.21.15.104:0/1476930117 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bcc104740 0x7f3bcc104ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 -- 172.21.15.104:0/1476930117 >> 172.21.15.104:0/1476930117 conn(0x7f3bcc0feae0 msgr2=0x7f3bcc100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 -- 172.21.15.104:0/1476930117 shutdown_connections 2024-05-08T05:44:38.515 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 -- 172.21.15.104:0/1476930117 wait complete. 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 Processor -- start 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 -- start start 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bcc103540 0x7f3bcc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bcc104740 0x7f3bcc199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.513+0000 7f3bd1808700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd1808700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10f030 con 0x7f3bcc103540 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd1808700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10eeb0 con 0x7f3bcc104740 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd1808700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3bcc10f1b0 con 0x7f3bcc1050e0 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc1a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:47828/0 (socket says 172.21.15.104:47828) 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 -- 172.21.15.104:0/3395715585 learned_addr learned my addr 172.21.15.104:0/3395715585 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcaffd700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bcc103540 0x7f3bcc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.516 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bca7fc700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bcc104740 0x7f3bcc199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 -- 172.21.15.104:0/3395715585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bcc104740 msgr2=0x7f3bcc199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bcc104740 0x7f3bcc199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 -- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bcc103540 msgr2=0x7f3bcc1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bcc103540 0x7f3bcc1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 -- 172.21.15.104:0/3395715585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bcc1a1300 con 0x7f3bcc1050e0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bcb7fe700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc1a0c60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00d9b0 tx=0x7f3bbc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc017070 con 0x7f3bcc1050e0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bcc1a15f0 con 0x7f3bcc1050e0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bbc005020 con 0x7f3bcc1050e0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc009990 con 0x7f3bcc1050e0 2024-05-08T05:44:38.517 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.514+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bcc1a1b60 con 0x7f3bcc1050e0 2024-05-08T05:44:38.518 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.515+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bcc06a190 con 0x7f3bcc1050e0 2024-05-08T05:44:38.519 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.517+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7f3bbc009b90 con 0x7f3bcc1050e0 2024-05-08T05:44:38.519 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.517+0000 7f3bd0806700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3bb80b27f0 0x7f3bb80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:44:38.519 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.517+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f3bbc0e0840 con 0x7f3bcc1050e0 2024-05-08T05:44:38.519 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.517+0000 7f3bcaffd700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3bb80b27f0 0x7f3bb80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:44:38.519 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.518+0000 7f3bcaffd700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3bb80b27f0 0x7f3bb80b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009fd0 tx=0x7f3bc400b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:44:38.522 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.522+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3bbc020020 con 0x7f3bcc1050e0 2024-05-08T05:44:38.734 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:38.734+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3bcc0638e0 con 0x7f3bb80b27f0 2024-05-08T05:44:39.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:38 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi089 2024-05-08T05:44:39.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:38 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi089 to 17088M 2024-05-08T05:44:39.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:38 smithi089 ceph-mon[34495]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:38 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi089 2024-05-08T05:44:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:38 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi089 to 17088M 2024-05-08T05:44:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:38 smithi104 ceph-mon[34559]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:39.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:38 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi089 2024-05-08T05:44:39.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:38 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi089 to 17088M 2024-05-08T05:44:39.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:38 smithi003 ceph-mon[30137]: pgmap v65: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:44:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:39 smithi089 ceph-mon[34495]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:44:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:39 smithi089 ceph-mon[34495]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:39 smithi089 ceph-mon[34495]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:39 smithi104 ceph-mon[34559]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:44:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:39 smithi104 ceph-mon[34559]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:39 smithi104 ceph-mon[34559]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:39 smithi003 ceph-mon[30137]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:44:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:39 smithi003 ceph-mon[30137]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:44:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:44:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:44:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:39 smithi003 ceph-mon[30137]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/2773369331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]: dispatch 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]: dispatch 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]': finished 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:44:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:42 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/654647006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/2773369331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]: dispatch 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]: dispatch 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]': finished 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:44:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:42 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/654647006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/2773369331' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]: dispatch 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]: dispatch 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c61371f5-cc2f-45a6-88b2-eefe6596c9b5"}]': finished 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:44:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:42 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/654647006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:44:44.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:44 smithi089 ceph-mon[34495]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:44 smithi104 ceph-mon[34559]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:44.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:44 smithi003 ceph-mon[30137]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:46.495 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:46 smithi104 ceph-mon[34559]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:46 smithi089 ceph-mon[34495]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:46.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:46 smithi003 ceph-mon[30137]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:48.377 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:48 smithi104 ceph-mon[34559]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:48.507 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:48 smithi089 ceph-mon[34495]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:48 smithi003 ceph-mon[30137]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:49.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:44:49.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:49.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:49 smithi089 ceph-mon[34495]: Deploying daemon osd.2 on smithi104 2024-05-08T05:44:49.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:44:49.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:49.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:49 smithi104 ceph-mon[34559]: Deploying daemon osd.2 on smithi104 2024-05-08T05:44:49.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:44:49.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:49.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:49 smithi003 ceph-mon[30137]: Deploying daemon osd.2 on smithi104 2024-05-08T05:44:50.455 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:50 smithi104 ceph-mon[34559]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:50.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:50 smithi089 ceph-mon[34495]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:50.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:50 smithi003 ceph-mon[30137]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:52.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:52 smithi089 ceph-mon[34495]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:52.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:52 smithi104 ceph-mon[34559]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:52.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:52 smithi003 ceph-mon[30137]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:54 smithi104 ceph-mon[34559]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:54.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:54 smithi089 ceph-mon[34495]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:54.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:54.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:54.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:54.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:54 smithi003 ceph-mon[30137]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:54.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:54.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:54.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:56 smithi104 ceph-mon[34559]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:56 smithi089 ceph-mon[34495]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:56 smithi003 ceph-mon[30137]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:44:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:44:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:57.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.755+0000 7f3bd0806700 1 -- 172.21.15.104:0/3395715585 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bcc0638e0 con 0x7f3bb80b27f0 2024-05-08T05:44:57.756 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 2 on host 'smithi104' 2024-05-08T05:44:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3bb80b27f0 msgr2=0x7f3bb80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3bb80b27f0 0x7f3bb80b4cb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3bc4009fd0 tx=0x7f3bc400b7b0 comp rx=0 tx=0).stop 2024-05-08T05:44:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 msgr2=0x7f3bcc1a0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:44:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc1a0c60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc00d9b0 tx=0x7f3bbc00dcc0 comp rx=0 tx=0).stop 2024-05-08T05:44:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 shutdown_connections 2024-05-08T05:44:57.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3bcc1050e0 0x7f3bcc1a0c60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3bb80b27f0 0x7f3bb80b4cb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3bcc103540 0x7f3bcc1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 --2- 172.21.15.104:0/3395715585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3bcc104740 0x7f3bcc199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:44:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 >> 172.21.15.104:0/3395715585 conn(0x7f3bcc0feae0 msgr2=0x7f3bcc107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:44:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 shutdown_connections 2024-05-08T05:44:57.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:44:57.758+0000 7f3bd1808700 1 -- 172.21.15.104:0/3395715585 wait complete. 2024-05-08T05:44:58.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:58 smithi089 ceph-mon[34495]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:58.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:58 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:58.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:58 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:58.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:58 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:58.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:58 smithi089 ceph-mon[34495]: from='osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:44:58.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:58 smithi089 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:44:58.443 DEBUG:teuthology.orchestra.run.smithi104:osd.2> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T05:44:58.445 INFO:tasks.cephadm:Deploying osd.3 on smithi003 with /dev/vg_nvme/lv_3... 2024-05-08T05:44:58.446 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:44:58.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:58 smithi003 ceph-mon[30137]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:58.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:58 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:58.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:58 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:58.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:58 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:58.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:58 smithi003 ceph-mon[30137]: from='osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:44:58.478 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:58 smithi003 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:44:58.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:58 smithi104 ceph-mon[34559]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:44:58.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:58 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:58.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:58 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:44:58.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:58 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:44:58.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:58 smithi104 ceph-mon[34559]: from='osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:44:58.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:58 smithi104 ceph-mon[34559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:44:58.703 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:44:59.964 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:59 smithi104 ceph-mon[34559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:44:59.964 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:59 smithi104 ceph-mon[34559]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:44:59.964 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:59 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:44:59.964 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:59 smithi104 ceph-mon[34559]: from='osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:44:59.964 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:59 smithi104 ceph-mon[34559]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:44:59.965 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:44:59 smithi104 ceph-mon[34559]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:59 smithi089 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:45:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:59 smithi089 ceph-mon[34495]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:45:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:59 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:59 smithi089 ceph-mon[34495]: from='osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:45:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:59 smithi089 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:45:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:44:59 smithi089 ceph-mon[34495]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:00.076 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:59 smithi003 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:45:00.076 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:59 smithi003 ceph-mon[30137]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:45:00.077 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:59 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:00.077 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:59 smithi003 ceph-mon[30137]: from='osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:45:00.077 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:59 smithi003 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:45:00.077 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:44:59 smithi003 ceph-mon[30137]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:00.271 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2[39728]: 2024-05-08T05:45:00.190+0000 7ff974605700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T05:45:00.272 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2[39728]: 2024-05-08T05:45:00.195+0000 7ff96c3f0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:45:00.658 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:45:00.658 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:45:00.658 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-05-08T05:45:00.658 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-05-08T05:45:00.658 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119008 s, 881 MB/s 2024-05-08T05:45:00.658 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-05-08T05:45:00.953 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:00 smithi003 ceph-mon[30137]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:45:00.954 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:00 smithi003 ceph-mon[30137]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:45:00.954 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:00.954 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:00.954 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:00 smithi003 ceph-mon[30137]: from='osd.2 ' entity='osd.2' 2024-05-08T05:45:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:00 smithi089 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:45:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:00 smithi089 ceph-mon[34495]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:45:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:00 smithi089 ceph-mon[34495]: from='osd.2 ' entity='osd.2' 2024-05-08T05:45:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:00 smithi104 ceph-mon[34559]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:45:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:00 smithi104 ceph-mon[34559]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:45:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:00 smithi104 ceph-mon[34559]: from='osd.2 ' entity='osd.2' 2024-05-08T05:45:01.315 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_3 2024-05-08T05:45:01.605 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:45:02.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:45:02.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:45:02.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:02.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:02.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] boot 2024-05-08T05:45:02.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:45:02.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:02.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:02.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi104 2024-05-08T05:45:02.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:02.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi104 to 17088M 2024-05-08T05:45:02.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:02.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:02.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] boot 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:02.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:02.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi104 2024-05-08T05:45:02.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.025 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:02.025 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi104 to 17088M 2024-05-08T05:45:02.025 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.025 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:02.025 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:02.025 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.059 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:45:02.059 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:45:02.059 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: osd.2 [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] boot 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi104 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi104 to 17088M 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:02.060 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:03.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:03.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 ceph-mon[30137]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:45:03.295 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:03.295 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:03.296 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.296 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49187]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.296 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49187]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 -- 172.21.15.3:0/740936870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08101d70 msgr2=0x7fce08102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 --2- 172.21.15.3:0/740936870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08101d70 0x7fce08102190 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fce04009990 tx=0x7fce04009ca0 comp rx=0 tx=0).stop 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 -- 172.21.15.3:0/740936870 shutdown_connections 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 --2- 172.21.15.3:0/740936870 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce08103090 0x7fce08103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 --2- 172.21.15.3:0/740936870 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08101d70 0x7fce08102190 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 --2- 172.21.15.3:0/740936870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce081026d0 0x7fce08102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 -- 172.21.15.3:0/740936870 >> 172.21.15.3:0/740936870 conn(0x7fce080fd990 msgr2=0x7fce080ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 -- 172.21.15.3:0/740936870 shutdown_connections 2024-05-08T05:45:03.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.302+0000 7fce0ff68700 1 -- 172.21.15.3:0/740936870 wait complete. 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 Processor -- start 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 -- start start 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce08101d70 0x7fce0819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 0x7fce0819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08103090 0x7fce081a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fce08111140 con 0x7fce081026d0 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fce08110fc0 con 0x7fce08103090 2024-05-08T05:45:03.305 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.303+0000 7fce0ff68700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fce081112c0 con 0x7fce08101d70 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0d503700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 0x7fce0819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0d503700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 0x7fce0819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42792/0 (socket says 172.21.15.3:42792) 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0d503700 1 -- 172.21.15.3:0/1276084975 learned_addr learned my addr 172.21.15.3:0/1276084975 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce08101d70 0x7fce0819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0e505700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08103090 0x7fce081a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 -- 172.21.15.3:0/1276084975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08103090 msgr2=0x7fce081a2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08103090 0x7fce081a2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 -- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 msgr2=0x7fce0819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 0x7fce0819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 -- 172.21.15.3:0/1276084975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce081a3440 con 0x7fce08101d70 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0d503700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 0x7fce0819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.304+0000 7fce0dd04700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce08101d70 0x7fce0819b7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce04015000 tx=0x7fce040057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:03.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.305+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04026070 con 0x7fce08101d70 2024-05-08T05:45:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.305+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce04021410 con 0x7fce08101d70 2024-05-08T05:45:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.305+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce081a36d0 con 0x7fce08101d70 2024-05-08T05:45:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.305+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04017650 con 0x7fce08101d70 2024-05-08T05:45:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.305+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081a3b20 con 0x7fce08101d70 2024-05-08T05:45:03.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.306+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0806a190 con 0x7fce08101d70 2024-05-08T05:45:03.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.308+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 149595+0+0 (secure 0 0 0) 0x7fce04029070 con 0x7fce08101d70 2024-05-08T05:45:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.308+0000 7fcdfeffd700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.309+0000 7fce0d503700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.309+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7fce040e1d70 con 0x7fce08101d70 2024-05-08T05:45:03.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.309+0000 7fce0d503700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8011630 tx=0x7fcdf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:03.313 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.313+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce04030080 con 0x7fce08101d70 2024-05-08T05:45:03.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:03.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 ceph-mon[34559]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:45:03.499 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:03.499 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:03.499 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.499 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43615]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.499 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43615]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.500 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:03.500 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.500 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43618]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.500 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43618]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:03.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 ceph-mon[34495]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:45:03.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:45:03.513 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43271]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:03.513 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43271]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.513 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43271]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.513 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43271]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.513 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43274]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:03.513 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43274]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.514 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43274]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.514 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43274]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:03.548+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fce080638e0 con 0x7fcdf40b27f0 2024-05-08T05:45:03.549 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:03.549 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.549 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49191]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.549 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49191]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49213]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:03.549 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49213]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49213]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49213]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:03.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49216]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.550 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:03 smithi003 sudo[49216]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43621]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:03.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43621]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43621]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43621]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:03.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43624]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:03.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43624]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:03.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43624]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:03.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:03 smithi104 sudo[43624]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:04.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43277]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43277]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43280]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43280]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:45:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:03 smithi089 sudo[43280]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.443 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:04.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:04.445 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:04 smithi003 ceph-mon[30137]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:45:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:45:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:04.515 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:04 smithi089 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:45:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:45:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:45:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:45:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:45:04.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:45:04.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:04.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:04.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:04.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:04.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:04 smithi104 ceph-mon[34559]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:45:05.229 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:05.229+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fce04009d10 con 0x7fce08101d70 2024-05-08T05:45:05.363 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:05 smithi003 ceph-mon[30137]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:45:05.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:05 smithi089 ceph-mon[34495]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:45:05.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:05 smithi104 ceph-mon[34559]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:45:06.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:06 smithi089 ceph-mon[34495]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:06.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:06 smithi089 ceph-mon[34495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:45:06.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:06 smithi089 ceph-mon[34495]: Cluster is now healthy 2024-05-08T05:45:06.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:06 smithi089 ceph-mon[34495]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:45:06.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:06 smithi104 ceph-mon[34559]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:06.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:06 smithi104 ceph-mon[34559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:45:06.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:06 smithi104 ceph-mon[34559]: Cluster is now healthy 2024-05-08T05:45:06.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:06 smithi104 ceph-mon[34559]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:45:06.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:06 smithi003 ceph-mon[30137]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:06.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:06 smithi003 ceph-mon[30137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:45:06.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:06 smithi003 ceph-mon[30137]: Cluster is now healthy 2024-05-08T05:45:06.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:06 smithi003 ceph-mon[30137]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:45:07.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:07 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1736617137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08946cb5-3b97-40ee-a729-6c9a386aeeb8"}]: dispatch 2024-05-08T05:45:07.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:07 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1736617137' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08946cb5-3b97-40ee-a729-6c9a386aeeb8"}]': finished 2024-05-08T05:45:07.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:07 smithi089 ceph-mon[34495]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:45:07.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:07.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:07 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1736617137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08946cb5-3b97-40ee-a729-6c9a386aeeb8"}]: dispatch 2024-05-08T05:45:07.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:07 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1736617137' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08946cb5-3b97-40ee-a729-6c9a386aeeb8"}]': finished 2024-05-08T05:45:07.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:07 smithi104 ceph-mon[34559]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:45:07.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:07 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1736617137' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08946cb5-3b97-40ee-a729-6c9a386aeeb8"}]: dispatch 2024-05-08T05:45:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:07 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1736617137' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08946cb5-3b97-40ee-a729-6c9a386aeeb8"}]': finished 2024-05-08T05:45:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:07 smithi003 ceph-mon[30137]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:45:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:08.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:08 smithi089 ceph-mon[34495]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:08.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:08 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3212984048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:08.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:08 smithi104 ceph-mon[34559]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:08.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:08 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3212984048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:08 smithi003 ceph-mon[30137]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:08 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3212984048' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:10.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:10 smithi089 ceph-mon[34495]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:10.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:10 smithi104 ceph-mon[34559]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:10.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:10 smithi003 ceph-mon[30137]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:12.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:12 smithi003 ceph-mon[30137]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:12.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:12 smithi089 ceph-mon[34495]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:12.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:12 smithi104 ceph-mon[34559]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:14.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:14 smithi003 ceph-mon[30137]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:14.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:45:14.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:14 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:14.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:14 smithi089 ceph-mon[34495]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:14.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:45:14.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:14 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:14.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:14 smithi104 ceph-mon[34559]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:14.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:45:14.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:14 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:15.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:15 smithi089 ceph-mon[34495]: Deploying daemon osd.3 on smithi003 2024-05-08T05:45:15.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:15 smithi104 ceph-mon[34559]: Deploying daemon osd.3 on smithi003 2024-05-08T05:45:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:15 smithi003 ceph-mon[30137]: Deploying daemon osd.3 on smithi003 2024-05-08T05:45:16.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:16 smithi089 ceph-mon[34495]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:16.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:16 smithi104 ceph-mon[34559]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:16.607 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:16 smithi003 ceph-mon[30137]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:18.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:18 smithi089 ceph-mon[34495]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:18.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:18 smithi104 ceph-mon[34559]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:18.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:18 smithi003 ceph-mon[30137]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:20.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:20 smithi089 ceph-mon[34495]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:20.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:20.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:20.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:20.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:20 smithi104 ceph-mon[34559]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:20.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:20.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:20.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:20.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:20 smithi003 ceph-mon[30137]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:20.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:20.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:20.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:22.358 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:22 smithi003 ceph-mon[30137]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:22.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:22 smithi104 ceph-mon[34559]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:22.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:22 smithi089 ceph-mon[34495]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:23.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.611+0000 7fcdfeffd700 1 -- 172.21.15.3:0/1276084975 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fce080638e0 con 0x7fcdf40b27f0 2024-05-08T05:45:23.613 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 3 on host 'smithi003' 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.614+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcdf40b27f0 msgr2=0x7fcdf40b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.614+0000 7fce0ff68700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8011630 tx=0x7fcdf8009260 comp rx=0 tx=0).stop 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce08101d70 msgr2=0x7fce0819b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce08101d70 0x7fce0819b7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce04015000 tx=0x7fce040057c0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 shutdown_connections 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fce08103090 0x7fce081a2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fce08101d70 0x7fce0819b7e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fcdf40b27f0 0x7fcdf40b4cb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 --2- 172.21.15.3:0/1276084975 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fce081026d0 0x7fce0819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 >> 172.21.15.3:0/1276084975 conn(0x7fce080fd990 msgr2=0x7fce080ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 shutdown_connections 2024-05-08T05:45:23.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:45:23.615+0000 7fce0ff68700 1 -- 172.21.15.3:0/1276084975 wait complete. 2024-05-08T05:45:24.293 DEBUG:teuthology.orchestra.run.smithi003:osd.3> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T05:45:24.295 INFO:tasks.cephadm:Deploying osd.4 on smithi089 with /dev/vg_nvme/lv_3... 2024-05-08T05:45:24.295 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:45:24.540 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:45:24.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:24 smithi089 ceph-mon[34495]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:24.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:24 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.855 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:24 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.856 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:24 smithi089 ceph-mon[34495]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:45:24.856 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:24 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.856 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:24 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:24 smithi003 ceph-mon[30137]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:24 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:24 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:24 smithi003 ceph-mon[30137]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:45:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:24 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:24 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:24 smithi104 ceph-mon[34559]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:24 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:24 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:24 smithi104 ceph-mon[34559]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:45:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:24 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:24 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:25.876 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:25 smithi089 ceph-mon[34495]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:45:25.876 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:25 smithi089 ceph-mon[34495]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:45:25.876 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:25 smithi089 ceph-mon[34495]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:45:25.876 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:25 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:25.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:25 smithi003 ceph-mon[30137]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:45:25.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:25 smithi003 ceph-mon[30137]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:45:25.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:25 smithi003 ceph-mon[30137]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:45:25.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:25 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:25 smithi104 ceph-mon[34559]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:45:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:25 smithi104 ceph-mon[34559]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:45:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:25 smithi104 ceph-mon[34559]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:45:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:25 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:26.349 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 05:45:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T05:45:26.051+0000 7f82f8c94700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T05:45:26.349 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 05:45:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T05:45:26.059+0000 7f82f0a7f700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:45:26.605 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:45:26.606 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:45:26.606 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2024-05-08T05:45:26.606 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2024-05-08T05:45:26.606 INFO:teuthology.orchestra.run.smithi089.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0126094 s, 832 MB/s 2024-05-08T05:45:26.606 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2024-05-08T05:45:26.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:26 smithi089 ceph-mon[34495]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:26.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:26 smithi089 ceph-mon[34495]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:45:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:26 smithi089 ceph-mon[34495]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:45:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:26 smithi089 ceph-mon[34495]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' 2024-05-08T05:45:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:26 smithi003 ceph-mon[30137]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:26 smithi003 ceph-mon[30137]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:45:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:26 smithi003 ceph-mon[30137]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:45:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:26 smithi003 ceph-mon[30137]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' 2024-05-08T05:45:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:26 smithi104 ceph-mon[34559]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:26 smithi104 ceph-mon[34559]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:45:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:26 smithi104 ceph-mon[34559]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:45:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:26 smithi104 ceph-mon[34559]: from='osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148]' entity='osd.3' 2024-05-08T05:45:27.237 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_3 2024-05-08T05:45:27.497 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:45:27.897 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:27 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:45:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:27 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:45:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:27 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:27 smithi003 ceph-mon[30137]: osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] boot 2024-05-08T05:45:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:27 smithi003 ceph-mon[30137]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:45:27.898 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:27 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:27.931 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:27 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:45:27.931 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:27 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:45:27.931 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:27 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:27.931 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:27 smithi089 ceph-mon[34495]: osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] boot 2024-05-08T05:45:27.931 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:27 smithi089 ceph-mon[34495]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:45:27.931 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:27 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:28.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:27 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:45:28.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:27 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:45:28.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:27 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:28.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:27 smithi104 ceph-mon[34559]: osd.3 [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] boot 2024-05-08T05:45:28.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:27 smithi104 ceph-mon[34559]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:45:28.024 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:27 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:45:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:45:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:28.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.915 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:28.915 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:45:28.915 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.915 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.915 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:28.916 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:45:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:45:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:29.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 -- 172.21.15.89:0/1146758760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c102400 msgr2=0x7f597c102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 --2- 172.21.15.89:0/1146758760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c102400 0x7f597c102820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f59700099c0 tx=0x7f5970009cd0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 -- 172.21.15.89:0/1146758760 shutdown_connections 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 --2- 172.21.15.89:0/1146758760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c103fa0 0x7f597c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 --2- 172.21.15.89:0/1146758760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c102400 0x7f597c102820 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 --2- 172.21.15.89:0/1146758760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f597c103600 0x7f597c103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 -- 172.21.15.89:0/1146758760 >> 172.21.15.89:0/1146758760 conn(0x7f597c0fd9a0 msgr2=0x7f597c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 -- 172.21.15.89:0/1146758760 shutdown_connections 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.156+0000 7f5982b91700 1 -- 172.21.15.89:0/1146758760 wait complete. 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 Processor -- start 2024-05-08T05:45:29.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 -- start start 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f597c102400 0x7f597c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 0x7f597c199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c103fa0 0x7f597c1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10de90 con 0x7f597c103fa0 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10dd10 con 0x7f597c103600 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f5982b91700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f597c10e010 con 0x7f597c102400 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 0x7f597c199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 0x7f597c199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50276/0 (socket says 172.21.15.89:50276) 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 -- 172.21.15.89:0/87057972 learned_addr learned my addr 172.21.15.89:0/87057972 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f598092d700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f597c102400 0x7f597c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f598112e700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c103fa0 0x7f597c1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 -- 172.21.15.89:0/87057972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f597c102400 msgr2=0x7f597c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f597c102400 0x7f597c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 -- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c103fa0 msgr2=0x7f597c1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c103fa0 0x7f597c1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.157+0000 7f597bfff700 1 -- 172.21.15.89:0/87057972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f597c104820 con 0x7f597c103600 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.158+0000 7f597bfff700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 0x7f597c199c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f596800d980 tx=0x7f596800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.158+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968017070 con 0x7f597c103600 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.158+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f597c104b10 con 0x7f597c103600 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.158+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5968005020 con 0x7f597c103600 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.158+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5968009b50 con 0x7f597c103600 2024-05-08T05:45:29.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.158+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f597c105000 con 0x7f597c103600 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.159+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f597c06a190 con 0x7f597c103600 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.160+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f59680051d0 con 0x7f597c103600 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.161+0000 7f5979ffb700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f596c0b2880 0x7f596c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.161+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f59680e0f90 con 0x7f597c103600 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.161+0000 7f598092d700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f596c0b2880 0x7f596c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:29.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.161+0000 7f598092d700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f596c0b2880 0x7f596c0b4d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5970000c00 tx=0x7f5970005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:29.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.165+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5968020020 con 0x7f597c103600 2024-05-08T05:45:29.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:29.374+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f597c06b890 con 0x7f596c0b2880 2024-05-08T05:45:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:29 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi003 2024-05-08T05:45:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:29 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi003 to 8546M 2024-05-08T05:45:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:29 smithi003 ceph-mon[30137]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:45:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:29 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:29 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:29 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:29.985 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:29 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi003 2024-05-08T05:45:29.985 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:29 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi003 to 8546M 2024-05-08T05:45:29.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:29 smithi089 ceph-mon[34495]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:45:29.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:29 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:29.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:29 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:29.986 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:29 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:29 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi003 2024-05-08T05:45:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:29 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi003 to 8546M 2024-05-08T05:45:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:29 smithi104 ceph-mon[34559]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:45:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:29 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:29 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:29 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:30 smithi003 ceph-mon[30137]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:30 smithi003 ceph-mon[30137]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:30.927 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:30 smithi089 ceph-mon[34495]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:30.928 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:30 smithi089 ceph-mon[34495]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:30 smithi104 ceph-mon[34559]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:30 smithi104 ceph-mon[34559]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:32 smithi089 ceph-mon[34495]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:32 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/3581161032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]: dispatch 2024-05-08T05:45:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:32 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]: dispatch 2024-05-08T05:45:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:32 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]': finished 2024-05-08T05:45:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:32 smithi089 ceph-mon[34495]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:45:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:32 smithi003 ceph-mon[30137]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:32 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/3581161032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]: dispatch 2024-05-08T05:45:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:32 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]: dispatch 2024-05-08T05:45:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:32 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]': finished 2024-05-08T05:45:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:32 smithi003 ceph-mon[30137]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:45:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:32 smithi104 ceph-mon[34559]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:32 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/3581161032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]: dispatch 2024-05-08T05:45:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:32 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]: dispatch 2024-05-08T05:45:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:32 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c7f59b73-f06d-4487-ad40-019eb1a6b470"}]': finished 2024-05-08T05:45:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:32 smithi104 ceph-mon[34559]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:45:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:33.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:33 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/3414447233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:33 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/3414447233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:33 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/3414447233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:34 smithi003 ceph-mon[30137]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:45:35.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:34 smithi089 ceph-mon[34495]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:45:35.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:34 smithi104 ceph-mon[34559]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:45:36.851 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:36 smithi089 ceph-mon[34495]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:45:36.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:36 smithi003 ceph-mon[30137]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:45:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:36 smithi104 ceph-mon[34559]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:45:38.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:38 smithi089 ceph-mon[34495]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:45:38.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:38 smithi003 ceph-mon[30137]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:45:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:38 smithi104 ceph-mon[34559]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:45:39.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:45:39.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:45:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:45:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:40 smithi003 ceph-mon[30137]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:45:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:40 smithi003 ceph-mon[30137]: Deploying daemon osd.4 on smithi089 2024-05-08T05:45:41.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:40 smithi089 ceph-mon[34495]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:45:41.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:40 smithi089 ceph-mon[34495]: Deploying daemon osd.4 on smithi089 2024-05-08T05:45:41.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:40 smithi104 ceph-mon[34559]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:45:41.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:40 smithi104 ceph-mon[34559]: Deploying daemon osd.4 on smithi089 2024-05-08T05:45:43.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:42 smithi089 ceph-mon[34495]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:45:43.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:42 smithi104 ceph-mon[34559]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:45:43.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:42 smithi003 ceph-mon[30137]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:45:44.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:44 smithi089 ceph-mon[34495]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T05:45:45.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:44 smithi104 ceph-mon[34559]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T05:45:45.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:44 smithi003 ceph-mon[30137]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-05-08T05:45:46.098 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:46.098 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:46.098 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:46.098 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:46 smithi104 ceph-mon[34559]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:46 smithi089 ceph-mon[34495]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:46.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:46.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:46.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:46.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:46 smithi003 ceph-mon[30137]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:48.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:48 smithi089 ceph-mon[34495]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:48.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:48 smithi104 ceph-mon[34559]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:48.596 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.596+0000 7f5979ffb700 1 -- 172.21.15.89:0/87057972 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f597c06b890 con 0x7f596c0b2880 2024-05-08T05:45:48.661 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 4 on host 'smithi089' 2024-05-08T05:45:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f596c0b2880 msgr2=0x7f596c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f596c0b2880 0x7f596c0b4d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5970000c00 tx=0x7f5970005fb0 comp rx=0 tx=0).stop 2024-05-08T05:45:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 msgr2=0x7f597c199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:48.662 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 0x7f597c199c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f596800d980 tx=0x7f596800dc90 comp rx=0 tx=0).stop 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 shutdown_connections 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f596c0b2880 0x7f596c0b4d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f597c103fa0 0x7f597c1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f597c102400 0x7f597c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 --2- 172.21.15.89:0/87057972 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f597c103600 0x7f597c199c00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.599+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 >> 172.21.15.89:0/87057972 conn(0x7f597c0fd9a0 msgr2=0x7f597c06b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.600+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 shutdown_connections 2024-05-08T05:45:48.663 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:45:48.600+0000 7f5982b91700 1 -- 172.21.15.89:0/87057972 wait complete. 2024-05-08T05:45:48.664 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:48 smithi003 ceph-mon[30137]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:49.207 DEBUG:teuthology.orchestra.run.smithi089:osd.4> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T05:45:49.209 INFO:tasks.cephadm:Deploying osd.5 on smithi104 with /dev/vg_nvme/lv_3... 2024-05-08T05:45:49.210 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:45:49.455 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:45:49.836 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.836 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.836 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:49 smithi104 ceph-mon[34559]: from='osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:45:49.836 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:49 smithi104 ceph-mon[34559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:45:49.837 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.837 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:49 smithi003 ceph-mon[30137]: from='osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:45:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:49 smithi003 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:45:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:49 smithi089 ceph-mon[34495]: from='osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:45:49.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:49 smithi089 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:45:49.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:49.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:50.828 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:50 smithi089 ceph-mon[34495]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:50.828 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:50 smithi089 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:45:50.828 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:50 smithi089 ceph-mon[34495]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:45:50.828 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:50.828 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:50 smithi089 ceph-mon[34495]: from='osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:45:50.828 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:50 smithi089 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:45:50.829 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 05:45:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T05:45:50.573+0000 7f879f255700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T05:45:50.829 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 05:45:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T05:45:50.578+0000 7f879b048700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:45:50.847 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:50 smithi104 ceph-mon[34559]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:50.847 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:50 smithi104 ceph-mon[34559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:45:50.847 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:50 smithi104 ceph-mon[34559]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:45:50.847 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:50.847 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:50 smithi104 ceph-mon[34559]: from='osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:45:50.847 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:50 smithi104 ceph-mon[34559]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:45:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:50 smithi003 ceph-mon[30137]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:50 smithi003 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:45:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:50 smithi003 ceph-mon[30137]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:45:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:50 smithi003 ceph-mon[30137]: from='osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:45:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:50 smithi003 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:45:51.437 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:45:51.438 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:45:51.438 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-08T05:45:51.438 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-08T05:45:51.438 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140205 s, 748 MB/s 2024-05-08T05:45:51.438 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-08T05:45:51.822 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:51 smithi104 ceph-mon[34559]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:45:51.822 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:51 smithi104 ceph-mon[34559]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:45:51.822 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:51.822 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:51 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:51.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:51 smithi003 ceph-mon[30137]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:45:51.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:51 smithi003 ceph-mon[30137]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:45:51.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:51.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:51 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:51.921 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:51 smithi089 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:45:51.921 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:51 smithi089 ceph-mon[34495]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:45:51.921 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:51.921 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:51 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:52.157 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-05-08T05:45:52.392 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] boot 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:45:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] boot 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:45:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:53.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:45:53.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:45:53.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:45:53.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: osd.4 [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] boot 2024-05-08T05:45:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:45:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:45:53.871 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi089 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi089 to 8544M 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:53.872 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:53.873 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi089 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi089 to 8544M 2024-05-08T05:45:53.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:53.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi089 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi089 to 8544M 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:45:54.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:54.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:45:54.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:45:54.085 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.083+0000 7f14c7686700 1 -- 172.21.15.104:0/3629721482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 msgr2=0x7f14c0103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.083+0000 7f14c7686700 1 --2- 172.21.15.104:0/3629721482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c0103a60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009990 tx=0x7f14b0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.083+0000 7f14c7686700 1 -- 172.21.15.104:0/3629721482 shutdown_connections 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 --2- 172.21.15.104:0/3629721482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c0103fa0 0x7f14c010a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 --2- 172.21.15.104:0/3629721482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14c0102400 0x7f14c0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 --2- 172.21.15.104:0/3629721482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c0103a60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- 172.21.15.104:0/3629721482 >> 172.21.15.104:0/3629721482 conn(0x7f14c00fd9a0 msgr2=0x7f14c00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- 172.21.15.104:0/3629721482 shutdown_connections 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- 172.21.15.104:0/3629721482 wait complete. 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 Processor -- start 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- start start 2024-05-08T05:45:54.086 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14c0102400 0x7f14c006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c0103fa0 0x7f14c01a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14c010dea0 con 0x7f14c0103fa0 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f14c010dd20 con 0x7f14c0102400 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c7686700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14c010e020 con 0x7f14c0103600 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c4c21700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c4c21700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:41414/0 (socket says 172.21.15.104:41414) 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.084+0000 7f14c4c21700 1 -- 172.21.15.104:0/1279054681 learned_addr learned my addr 172.21.15.104:0/1279054681 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c5422700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14c0102400 0x7f14c006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c4c21700 1 -- 172.21.15.104:0/1279054681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14c0102400 msgr2=0x7f14c006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c4c21700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14c0102400 0x7f14c006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c4c21700 1 -- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c0103fa0 msgr2=0x7f14c01a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c5c23700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c0103fa0 0x7f14c01a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:54.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c4c21700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c0103fa0 0x7f14c01a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c4c21700 1 -- 172.21.15.104:0/1279054681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c01a1340 con 0x7f14c0103600 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c4c21700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c019e430 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14b0006010 tx=0x7f14b00047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0027070 con 0x7f14c0103600 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c01a15d0 con 0x7f14c0103600 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b0004140 con 0x7f14c0103600 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0022e00 con 0x7f14c0103600 2024-05-08T05:45:54.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.085+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c01a1a80 con 0x7f14c0103600 2024-05-08T05:45:54.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.086+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c006a190 con 0x7f14c0103600 2024-05-08T05:45:54.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.087+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f14b002a070 con 0x7f14c0103600 2024-05-08T05:45:54.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.088+0000 7f14b67fc700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14ac0b2880 0x7f14ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:45:54.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.088+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7f14b00e24a0 con 0x7f14c0103600 2024-05-08T05:45:54.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.088+0000 7f14c5422700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14ac0b2880 0x7f14ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:45:54.089 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.089+0000 7f14c5422700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14ac0b2880 0x7f14ac0b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14bc005950 tx=0x7f14bc00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:45:54.092 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.092+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14b0031070 con 0x7f14c0103600 2024-05-08T05:45:54.305 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:45:54.305+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f14c00638e0 con 0x7f14ac0b2880 2024-05-08T05:45:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:54 smithi003 ceph-mon[30137]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:54.916 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:54 smithi104 ceph-mon[34559]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:54.917 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:54.917 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:54.917 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:55.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:54 smithi089 ceph-mon[34495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:55.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:45:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:45:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:45:55.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:55 smithi003 ceph-mon[30137]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:55.938 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:55 smithi104 ceph-mon[34559]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:55 smithi089 ceph-mon[34495]: from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:45:56.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:56 smithi003 ceph-mon[30137]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:56 smithi089 ceph-mon[34495]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:56 smithi104 ceph-mon[34559]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:57 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/664516865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]: dispatch 2024-05-08T05:45:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:57 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]: dispatch 2024-05-08T05:45:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:57 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]': finished 2024-05-08T05:45:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:57 smithi003 ceph-mon[30137]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:45:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:45:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:57 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/664516865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]: dispatch 2024-05-08T05:45:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:57 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]: dispatch 2024-05-08T05:45:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:57 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]': finished 2024-05-08T05:45:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:57 smithi089 ceph-mon[34495]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:45:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:45:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:57 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/664516865' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]: dispatch 2024-05-08T05:45:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:57 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]: dispatch 2024-05-08T05:45:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:57 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8e47b2-6008-416b-ba61-f08cf12933b9"}]': finished 2024-05-08T05:45:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:57 smithi104 ceph-mon[34559]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:45:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:45:58.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:58 smithi089 ceph-mon[34495]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:58.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:45:58 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/2730851950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:58 smithi003 ceph-mon[30137]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:58.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:45:58 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/2730851950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:45:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:58 smithi104 ceph-mon[34559]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:45:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:45:58 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/2730851950' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:00 smithi003 ceph-mon[30137]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:00.979 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:00 smithi104 ceph-mon[34559]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:00 smithi089 ceph-mon[34495]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:02 smithi003 ceph-mon[30137]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:02.967 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:02 smithi104 ceph-mon[34559]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:02 smithi089 ceph-mon[34495]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:04.875 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:04 smithi104 ceph-mon[34559]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:04.875 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:46:04.876 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:04 smithi003 ceph-mon[30137]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:46:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:05.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:04 smithi089 ceph-mon[34495]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:46:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:05.715 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:05 smithi104 ceph-mon[34559]: Deploying daemon osd.5 on smithi104 2024-05-08T05:46:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:05 smithi003 ceph-mon[30137]: Deploying daemon osd.5 on smithi104 2024-05-08T05:46:06.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:05 smithi089 ceph-mon[34495]: Deploying daemon osd.5 on smithi104 2024-05-08T05:46:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:06 smithi003 ceph-mon[30137]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:06.971 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:06 smithi104 ceph-mon[34559]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:07.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:06 smithi089 ceph-mon[34495]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:08.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:08 smithi104 ceph-mon[34559]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:08.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:08 smithi003 ceph-mon[30137]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:09.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:08 smithi089 ceph-mon[34495]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:10.736 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:10 smithi104 ceph-mon[34559]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:10.736 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:10.736 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:10.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:10 smithi003 ceph-mon[30137]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:11.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:10 smithi089 ceph-mon[34495]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:11.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:11.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:11.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:12 smithi003 ceph-mon[30137]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:12.968 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:12 smithi104 ceph-mon[34559]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:12 smithi089 ceph-mon[34495]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:13.575 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.574+0000 7f14b67fc700 1 -- 172.21.15.104:0/1279054681 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14c00638e0 con 0x7f14ac0b2880 2024-05-08T05:46:13.575 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 5 on host 'smithi104' 2024-05-08T05:46:13.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14ac0b2880 msgr2=0x7f14ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:13.577 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14ac0b2880 0x7f14ac0b4d40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14bc005950 tx=0x7f14bc00a300 comp rx=0 tx=0).stop 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 msgr2=0x7f14c019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c019e430 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14b0006010 tx=0x7f14b00047b0 comp rx=0 tx=0).stop 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 shutdown_connections 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14ac0b2880 0x7f14ac0b4d40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14c0103fa0 0x7f14c01a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14c0102400 0x7f14c006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 --2- 172.21.15.104:0/1279054681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14c0103600 0x7f14c019e430 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 >> 172.21.15.104:0/1279054681 conn(0x7f14c00fd9a0 msgr2=0x7f14c0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 shutdown_connections 2024-05-08T05:46:13.578 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:46:13.577+0000 7f14c7686700 1 -- 172.21.15.104:0/1279054681 wait complete. 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:14.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:46:14.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:13 smithi003 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:46:14.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:14.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:46:14.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:13 smithi089 ceph-mon[34495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:46:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:14.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:14.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:14.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:46:14.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:13 smithi104 ceph-mon[34559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:46:14.305 DEBUG:teuthology.orchestra.run.smithi104:osd.5> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T05:46:14.307 INFO:tasks.cephadm:Deploying osd.6 on smithi003 with /dev/vg_nvme/lv_2... 2024-05-08T05:46:14.307 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:46:14.568 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:46:15.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:15 smithi104 ceph-mon[34559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:46:15.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:15 smithi104 ceph-mon[34559]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:46:15.805 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:15.806 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:15 smithi104 ceph-mon[34559]: from='osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:46:15.806 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:15 smithi104 ceph-mon[34559]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:46:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:15 smithi003 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:46:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:15 smithi003 ceph-mon[30137]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:46:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:15 smithi003 ceph-mon[30137]: from='osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:46:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:15 smithi003 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:46:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:15 smithi089 ceph-mon[34495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:46:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:15 smithi089 ceph-mon[34495]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:46:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:15 smithi089 ceph-mon[34495]: from='osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:46:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:15 smithi089 ceph-mon[34495]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:46:16.099 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 05:46:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T05:46:16.024+0000 7f2442668700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T05:46:16.099 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 05:46:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T05:46:16.030+0000 7f243a453700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:46:16.740 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:46:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:46:16.741 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-05-08T05:46:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-05-08T05:46:16.741 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.015162 s, 692 MB/s 2024-05-08T05:46:16.741 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-05-08T05:46:16.767 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:16.767 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:46:16.767 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:46:16.767 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:16.768 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:16.768 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: from='osd.5 ' entity='osd.5' 2024-05-08T05:46:16.768 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] boot 2024-05-08T05:46:16.768 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T05:46:16.768 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:16 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: from='osd.5 ' entity='osd.5' 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] boot 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T05:46:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:16 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: from='osd.5 ' entity='osd.5' 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: osd.5 [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] boot 2024-05-08T05:46:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T05:46:17.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:16 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:46:17.566 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_2 2024-05-08T05:46:17.820 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:46:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:17 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:46:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:17 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:46:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:17 smithi003 ceph-mon[30137]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:46:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:17 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:46:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:17 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:46:18.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:17 smithi089 ceph-mon[34495]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:46:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:17 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:46:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:17 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:46:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:17 smithi104 ceph-mon[34559]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:46:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi104 2024-05-08T05:46:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi104 to 8544M 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:18.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:18 smithi003 ceph-mon[30137]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:46:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi104 2024-05-08T05:46:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:19.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi104 to 8544M 2024-05-08T05:46:19.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:19.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:19.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:18 smithi089 ceph-mon[34495]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi104 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi104 to 8544M 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:19.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:19.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:18 smithi104 ceph-mon[34559]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:46:19.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 -- 172.21.15.3:0/795000010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f41050a0 msgr2=0x7f55f410b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/795000010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f41050a0 0x7f55f410b950 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b0d0 tx=0x7f55ec00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:46:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 -- 172.21.15.3:0/795000010 shutdown_connections 2024-05-08T05:46:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/795000010 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f41050a0 0x7f55f410b950 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:19.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/795000010 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 0x7f55f4103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/795000010 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f4104700 0x7f55f4104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 -- 172.21.15.3:0/795000010 >> 172.21.15.3:0/795000010 conn(0x7f55f40feaa0 msgr2=0x7f55f4100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 -- 172.21.15.3:0/795000010 shutdown_connections 2024-05-08T05:46:19.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.579+0000 7f55f8ac6700 1 -- 172.21.15.3:0/795000010 wait complete. 2024-05-08T05:46:19.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 Processor -- start 2024-05-08T05:46:19.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 -- start start 2024-05-08T05:46:19.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 0x7f55f4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 0x7f55f4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f41050a0 0x7f55f41a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410ef80 con 0x7f55f4104700 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410ee00 con 0x7f55f4103500 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f8ac6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f55f410f100 con 0x7f55f41050a0 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 0x7f55f4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f1d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 0x7f55f4199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:55758/0 (socket says 172.21.15.3:55758) 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.580+0000 7f55f1d9b700 1 -- 172.21.15.3:0/3655743043 learned_addr learned my addr 172.21.15.3:0/3655743043 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f259c700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 0x7f55f4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f2d9d700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f41050a0 0x7f55f41a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f1d9b700 1 -- 172.21.15.3:0/3655743043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f41050a0 msgr2=0x7f55f41a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f1d9b700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f41050a0 0x7f55f41a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:19.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f1d9b700 1 -- 172.21.15.3:0/3655743043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 msgr2=0x7f55f4199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f1d9b700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 0x7f55f4199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f1d9b700 1 -- 172.21.15.3:0/3655743043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f41a11a0 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f2d9d700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f41050a0 0x7f55f41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f259c700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 0x7f55f4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f1d9b700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 0x7f55f4199a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c9e0 tx=0x7f55e800cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8007ac0 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f41a1490 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55e8007c60 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8018660 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.581+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f41a1a20 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.582+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f406a190 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.584+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f55e800f450 con 0x7f55f4104700 2024-05-08T05:46:19.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.584+0000 7f55e37fe700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55dc0b2880 0x7f55dc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.585+0000 7f55f259c700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55dc0b2880 0x7f55dc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.585+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f55e80e1790 con 0x7f55f4104700 2024-05-08T05:46:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.585+0000 7f55f259c700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55dc0b2880 0x7f55dc0b4d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55f4104560 tx=0x7f55e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:19.592 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.588+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55e8014030 con 0x7f55f4104700 2024-05-08T05:46:19.812 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:19.811+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f55f40638e0 con 0x7f55dc0b2880 2024-05-08T05:46:20.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:20 smithi003 ceph-mon[30137]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:20.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:20.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:20.718 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:20 smithi089 ceph-mon[34495]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:20 smithi104 ceph-mon[34559]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:21.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:21.852 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:21 smithi003 ceph-mon[30137]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:22.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:21 smithi089 ceph-mon[34495]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:21 smithi104 ceph-mon[34559]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:22 smithi003 ceph-mon[30137]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:23.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:22 smithi089 ceph-mon[34495]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:23.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:22 smithi104 ceph-mon[34559]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:23 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2040033846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ace0899-0a39-4753-a710-047f463b5a7f"}]: dispatch 2024-05-08T05:46:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:23 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2040033846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ace0899-0a39-4753-a710-047f463b5a7f"}]': finished 2024-05-08T05:46:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:23 smithi003 ceph-mon[30137]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T05:46:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:23 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:23 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2181497427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:23 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2040033846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ace0899-0a39-4753-a710-047f463b5a7f"}]: dispatch 2024-05-08T05:46:24.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:23 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2040033846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ace0899-0a39-4753-a710-047f463b5a7f"}]': finished 2024-05-08T05:46:24.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:23 smithi089 ceph-mon[34495]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T05:46:24.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:23 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:24.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:23 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2181497427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:23 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2040033846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7ace0899-0a39-4753-a710-047f463b5a7f"}]: dispatch 2024-05-08T05:46:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:23 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2040033846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ace0899-0a39-4753-a710-047f463b5a7f"}]': finished 2024-05-08T05:46:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:23 smithi104 ceph-mon[34559]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T05:46:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:23 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:23 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2181497427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:24 smithi003 ceph-mon[30137]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:46:25.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:24 smithi089 ceph-mon[34495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:46:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:24 smithi104 ceph-mon[34559]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:46:26.890 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:26 smithi003 ceph-mon[30137]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:46:27.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:26 smithi089 ceph-mon[34495]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:46:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:26 smithi104 ceph-mon[34559]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-05-08T05:46:28.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:28 smithi003 ceph-mon[30137]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T05:46:29.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:28 smithi089 ceph-mon[34495]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T05:46:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:28 smithi104 ceph-mon[34559]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-05-08T05:46:30.662 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:30 smithi003 ceph-mon[30137]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:31.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:30 smithi089 ceph-mon[34495]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:30 smithi104 ceph-mon[34559]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:31.833 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:46:31.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:46:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:46:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:32 smithi003 ceph-mon[30137]: Deploying daemon osd.6 on smithi003 2024-05-08T05:46:32.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:32 smithi003 ceph-mon[30137]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:33.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:32 smithi089 ceph-mon[34495]: Deploying daemon osd.6 on smithi003 2024-05-08T05:46:33.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:32 smithi089 ceph-mon[34495]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:32 smithi104 ceph-mon[34559]: Deploying daemon osd.6 on smithi003 2024-05-08T05:46:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:32 smithi104 ceph-mon[34559]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:46:34.871 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:34 smithi003 ceph-mon[30137]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:35.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:34 smithi089 ceph-mon[34495]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:35.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:34 smithi104 ceph-mon[34559]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:36.838 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:36 smithi003 ceph-mon[30137]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:37.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:36 smithi089 ceph-mon[34495]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:36 smithi104 ceph-mon[34559]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:38.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:38 smithi089 ceph-mon[34495]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:38.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:38.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:38.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:38.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:38 smithi104 ceph-mon[34559]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:38.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:38.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:38.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:38.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:38 smithi003 ceph-mon[30137]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:38.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:38.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:38.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:40.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:40 smithi003 ceph-mon[30137]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:40.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:40 smithi089 ceph-mon[34495]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:40.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:40 smithi104 ceph-mon[34559]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:41.376 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 6 on host 'smithi003' 2024-05-08T05:46:41.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.375+0000 7f55e37fe700 1 -- 172.21.15.3:0/3655743043 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55f40638e0 con 0x7f55dc0b2880 2024-05-08T05:46:41.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55dc0b2880 msgr2=0x7f55dc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:41.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55dc0b2880 0x7f55dc0b4d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f55f4104560 tx=0x7f55e4009260 comp rx=0 tx=0).stop 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 msgr2=0x7f55f4199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 0x7f55f4199a80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c9e0 tx=0x7f55e800cda0 comp rx=0 tx=0).stop 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 shutdown_connections 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f55f41050a0 0x7f55f41a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f55f4103500 0x7f55f4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55dc0b2880 0x7f55dc0b4d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 --2- 172.21.15.3:0/3655743043 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f55f4104700 0x7f55f4199a80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 >> 172.21.15.3:0/3655743043 conn(0x7f55f40feaa0 msgr2=0x7f55f4107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 shutdown_connections 2024-05-08T05:46:41.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:46:41.378+0000 7f55f8ac6700 1 -- 172.21.15.3:0/3655743043 wait complete. 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:41.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:46:42.133 DEBUG:teuthology.orchestra.run.smithi003:osd.6> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T05:46:42.136 INFO:tasks.cephadm:Deploying osd.7 on smithi089 with /dev/vg_nvme/lv_2... 2024-05-08T05:46:42.137 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:46:42.381 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:46:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:42 smithi003 ceph-mon[30137]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:42 smithi003 ceph-mon[30137]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:46:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:42 smithi003 ceph-mon[30137]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:46:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:42 smithi003 ceph-mon[30137]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:46:42.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:42.661 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 05:46:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T05:46:42.475+0000 7fcc2ae42700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T05:46:42.661 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 05:46:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T05:46:42.486+0000 7fcc26c35700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:46:42.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:42 smithi089 ceph-mon[34495]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:42.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:42 smithi089 ceph-mon[34495]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:46:42.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:42 smithi089 ceph-mon[34495]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:46:42.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:42 smithi089 ceph-mon[34495]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:46:42.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:42 smithi104 ceph-mon[34559]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:42 smithi104 ceph-mon[34559]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:46:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:42 smithi104 ceph-mon[34559]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:46:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:42 smithi104 ceph-mon[34559]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:46:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:43.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:43 smithi003 ceph-mon[30137]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:46:43.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:43 smithi003 ceph-mon[30137]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:46:43.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:43.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:43 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:43 smithi089 ceph-mon[34495]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:46:43.750 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:43 smithi089 ceph-mon[34495]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:46:43.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:43.751 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:43 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:43.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:43 smithi104 ceph-mon[34559]: from='osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:46:43.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:43 smithi104 ceph-mon[34559]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:46:43.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:43.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:43 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:46:44.580 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2024-05-08T05:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2024-05-08T05:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141879 s, 739 MB/s 2024-05-08T05:46:44.581 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] boot 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T05:46:44.581 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] boot 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T05:46:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:46:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:46:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:46:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: osd.6 [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] boot 2024-05-08T05:46:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T05:46:44.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:46:45.226 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_2 2024-05-08T05:46:45.463 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:46:45.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:46:45.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:45.776 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:46:45.777 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:46:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:46 smithi089 ceph-mon[34495]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:46.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:46 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi003 2024-05-08T05:46:46.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:46 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi003 to 5697M 2024-05-08T05:46:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:46 smithi104 ceph-mon[34559]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:46 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi003 2024-05-08T05:46:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:46 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi003 to 5697M 2024-05-08T05:46:46.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:46 smithi003 ceph-mon[30137]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:46.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:46 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi003 2024-05-08T05:46:46.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:46 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi003 to 5697M 2024-05-08T05:46:47.140 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 -- 172.21.15.89:0/1034925684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 msgr2=0x7ff898102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/1034925684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898102810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8880099c0 tx=0x7ff888009cd0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 -- 172.21.15.89:0/1034925684 shutdown_connections 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/1034925684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff898103f90 0x7ff89810a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/1034925684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898102810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/1034925684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff8981035f0 0x7ff898103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 -- 172.21.15.89:0/1034925684 >> 172.21.15.89:0/1034925684 conn(0x7ff8980fd990 msgr2=0x7ff8980ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 -- 172.21.15.89:0/1034925684 shutdown_connections 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 -- 172.21.15.89:0/1034925684 wait complete. 2024-05-08T05:46:47.141 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.140+0000 7ff89ca3a700 1 Processor -- start 2024-05-08T05:46:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 -- start start 2024-05-08T05:46:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.143 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8981035f0 0x7ff8981049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff898103f90 0x7ff898104f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810de80 con 0x7ff8981035f0 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810dd00 con 0x7ff8981023f0 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89ca3a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff89810e000 con 0x7ff898103f90 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898106380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40986/0 (socket says 172.21.15.89:40986) 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 -- 172.21.15.89:0/2375733769 learned_addr learned my addr 172.21.15.89:0/2375733769 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff896d9d700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff898103f90 0x7ff898104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff88dd9b700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8981035f0 0x7ff8981049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 -- 172.21.15.89:0/2375733769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff898103f90 msgr2=0x7ff898104f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff898103f90 0x7ff898104f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 -- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8981035f0 msgr2=0x7ff8981049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:46:47.144 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8981035f0 0x7ff8981049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 -- 172.21.15.89:0/2375733769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8981057c0 con 0x7ff8981023f0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff88dd9b700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8981035f0 0x7ff8981049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.141+0000 7ff89659c700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898106380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8880038c0 tx=0x7ff8880047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.142+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888027070 con 0x7ff8981023f0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.142+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff888004160 con 0x7ff8981023f0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.142+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff888022e20 con 0x7ff8981023f0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.142+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8981a37d0 con 0x7ff8981023f0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.142+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8981a3c80 con 0x7ff8981023f0 2024-05-08T05:46:47.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.143+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff89806a190 con 0x7ff8981023f0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.145+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff88802a070 con 0x7ff8981023f0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.145+0000 7ff88ffff700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff8840b2880 0x7ff8840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.146+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7ff8880e23e0 con 0x7ff8981023f0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.146+0000 7ff88dd9b700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff8840b2880 0x7ff8840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:46:47.146 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.146+0000 7ff88dd9b700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff8840b2880 0x7ff8840b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff88000bcf0 tx=0x7ff88000b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:46:47.149 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.149+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff888031080 con 0x7ff8981023f0 2024-05-08T05:46:47.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:46:47.360+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff89806c310 con 0x7ff8840b2880 2024-05-08T05:46:47.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:47.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:47.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:47.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:47.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:47.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:46:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:46:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:48 smithi089 ceph-mon[34495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:48 smithi089 ceph-mon[34495]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:48 smithi104 ceph-mon[34559]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:48 smithi104 ceph-mon[34559]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:48 smithi003 ceph-mon[30137]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:48 smithi003 ceph-mon[30137]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:46:50.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:50 smithi089 ceph-mon[34495]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:50.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/4025235225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]: dispatch 2024-05-08T05:46:50.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:50 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]: dispatch 2024-05-08T05:46:50.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:50 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]': finished 2024-05-08T05:46:50.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:50 smithi089 ceph-mon[34495]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T05:46:50.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:46:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:50 smithi104 ceph-mon[34559]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/4025235225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]: dispatch 2024-05-08T05:46:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:50 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]: dispatch 2024-05-08T05:46:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:50 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]': finished 2024-05-08T05:46:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:50 smithi104 ceph-mon[34559]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T05:46:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:46:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:50 smithi003 ceph-mon[30137]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:51.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/4025235225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]: dispatch 2024-05-08T05:46:51.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:50 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]: dispatch 2024-05-08T05:46:51.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:50 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13aa9f3f-8b0f-43e4-b328-653822cc6b28"}]': finished 2024-05-08T05:46:51.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:50 smithi003 ceph-mon[30137]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T05:46:51.014 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:46:51.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:51 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/3879439400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:51 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/3879439400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:51.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:51 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/3879439400' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:46:52.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:52 smithi089 ceph-mon[34495]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:52 smithi104 ceph-mon[34559]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:52 smithi003 ceph-mon[30137]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:54.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:54 smithi089 ceph-mon[34495]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:54 smithi104 ceph-mon[34559]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:54 smithi003 ceph-mon[30137]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:56 smithi104 ceph-mon[34559]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:56.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:56 smithi003 ceph-mon[30137]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:56 smithi089 ceph-mon[34495]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:57.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:46:57.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:46:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:46:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:46:58.889 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:58 smithi089 ceph-mon[34495]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:58.889 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:46:58 smithi089 ceph-mon[34495]: Deploying daemon osd.7 on smithi089 2024-05-08T05:46:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:58 smithi003 ceph-mon[30137]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:46:58 smithi003 ceph-mon[30137]: Deploying daemon osd.7 on smithi089 2024-05-08T05:46:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:58 smithi104 ceph-mon[34559]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:46:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:46:58 smithi104 ceph-mon[34559]: Deploying daemon osd.7 on smithi089 2024-05-08T05:47:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:00 smithi003 ceph-mon[30137]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:00.932 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:00 smithi089 ceph-mon[34495]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:00 smithi104 ceph-mon[34559]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:02.608 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:02 smithi089 ceph-mon[34495]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:02 smithi003 ceph-mon[30137]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:02 smithi104 ceph-mon[34559]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:04.141 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:03 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:04.141 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:03 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:04.141 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:03 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:04.141 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:03 smithi089 ceph-mon[34495]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:04.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:03 smithi003 ceph-mon[30137]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:03 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:03 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:03 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:03 smithi104 ceph-mon[34559]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:06.327 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.327+0000 7ff88ffff700 1 -- 172.21.15.89:0/2375733769 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff89806c310 con 0x7ff8840b2880 2024-05-08T05:47:06.328 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 7 on host 'smithi089' 2024-05-08T05:47:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff8840b2880 msgr2=0x7ff8840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.329 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff8840b2880 0x7ff8840b4d40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff88000bcf0 tx=0x7ff88000b2c0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 msgr2=0x7ff898106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898106380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8880038c0 tx=0x7ff8880047b0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 shutdown_connections 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff898103f90 0x7ff898104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff8981023f0 0x7ff898106380 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff8840b2880 0x7ff8840b4d40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 --2- 172.21.15.89:0/2375733769 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff8981035f0 0x7ff8981049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 >> 172.21.15.89:0/2375733769 conn(0x7ff8980fd990 msgr2=0x7ff89806bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 shutdown_connections 2024-05-08T05:47:06.330 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:47:06.329+0000 7ff89ca3a700 1 -- 172.21.15.89:0/2375733769 wait complete. 2024-05-08T05:47:06.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:06 smithi089 ceph-mon[34495]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:06.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:06 smithi104 ceph-mon[34559]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:06.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:06 smithi003 ceph-mon[30137]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:06.994 DEBUG:teuthology.orchestra.run.smithi089:osd.7> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T05:47:06.996 INFO:tasks.cephadm:Deploying osd.8 on smithi104 with /dev/vg_nvme/lv_2... 2024-05-08T05:47:06.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:47:07.245 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:47:07.565 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.565 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.565 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:07 smithi104 ceph-mon[34559]: from='osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:47:07.565 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:07 smithi104 ceph-mon[34559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:47:07.565 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.565 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:07 smithi003 ceph-mon[30137]: from='osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:47:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:07 smithi003 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:47:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.729 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:07 smithi089 ceph-mon[34495]: from='osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:47:07.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:07 smithi089 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:47:07.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:07.730 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:08.626 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:08 smithi104 ceph-mon[34559]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:08.626 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:08 smithi104 ceph-mon[34559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:47:08.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:08 smithi104 ceph-mon[34559]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:47:08.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:08 smithi104 ceph-mon[34559]: from='osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:47:08.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:08 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:08.627 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:08 smithi104 ceph-mon[34559]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:47:08.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:08 smithi003 ceph-mon[30137]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:08 smithi003 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:47:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:08 smithi003 ceph-mon[30137]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:47:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:08 smithi003 ceph-mon[30137]: from='osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:47:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:08 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:08 smithi003 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:47:08.700 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:08 smithi089 ceph-mon[34495]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:08.701 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:08 smithi089 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:47:08.701 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:08 smithi089 ceph-mon[34495]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:47:08.701 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:08 smithi089 ceph-mon[34495]: from='osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:47:08.701 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:08 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:08.701 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:08 smithi089 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:47:09.000 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 05:47:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T05:47:08.763+0000 7faf4e8cf700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T05:47:09.001 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 05:47:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T05:47:08.773+0000 7faf466ba700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:47:09.340 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:47:09.340 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:47:09.340 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-08T05:47:09.340 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-08T05:47:09.340 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123429 s, 850 MB/s 2024-05-08T05:47:09.340 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-08T05:47:09.651 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: from='osd.7 ' entity='osd.7' 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] boot 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T05:47:09.652 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:09 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: from='osd.7 ' entity='osd.7' 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] boot 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T05:47:09.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:09 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: from='osd.7 ' entity='osd.7' 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: osd.7 [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] boot 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T05:47:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:09 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:47:10.056 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-05-08T05:47:10.309 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:47:10.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:10 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:47:10.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:10 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:47:10.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:10 smithi089 ceph-mon[34495]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:10.457 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:10 smithi089 ceph-mon[34495]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:47:10.617 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:10 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:47:10.617 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:10 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:47:10.617 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:10 smithi104 ceph-mon[34559]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:10.617 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:10 smithi104 ceph-mon[34559]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:47:10.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:10 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:47:10.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:10 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:47:10.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:10 smithi003 ceph-mon[30137]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:47:10.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:10 smithi003 ceph-mon[30137]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:47:11.736 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi089 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi089 to 5696M 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:11.737 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:11.738 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi089 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi089 to 5696M 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:11.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi089 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi089 to 5696M 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:11.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/281304890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 msgr2=0x7f1d28104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/281304890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28104b80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009990 tx=0x7f1d18009ca0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/281304890 shutdown_connections 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/281304890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d281050c0 0x7f1d2810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/281304890 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d28103520 0x7f1d28103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/281304890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28104b80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/281304890 >> 172.21.15.104:0/281304890 conn(0x7f1d280feac0 msgr2=0x7f1d28100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:11.992 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/281304890 shutdown_connections 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/281304890 wait complete. 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.990+0000 7f1d2f9d2700 1 Processor -- start 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 -- start start 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d28103520 0x7f1d281996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d281050c0 0x7f1d281a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2810f010 con 0x7f1d281050c0 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2810ee90 con 0x7f1d28103520 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2810f190 con 0x7f1d28104720 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:39022/0 (socket says 172.21.15.104:39022) 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 -- 172.21.15.104:0/2765707819 learned_addr learned my addr 172.21.15.104:0/2765707819 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2df6f700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d281050c0 0x7f1d281a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.993 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2d76e700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d28103520 0x7f1d281996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 -- 172.21.15.104:0/2765707819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d28103520 msgr2=0x7f1d281996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d28103520 0x7f1d281996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 -- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d281050c0 msgr2=0x7f1d281a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d281050c0 0x7f1d281a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 -- 172.21.15.104:0/2765707819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d281a1300 con 0x7f1d28104720 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2d76e700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d28103520 0x7f1d281996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2cf6d700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28199be0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d180036f0 tx=0x7f1d180047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18027070 con 0x7f1d28104720 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d281a1590 con 0x7f1d28104720 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d18004100 con 0x7f1d28104720 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.991+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18022d10 con 0x7f1d28104720 2024-05-08T05:47:11.994 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.992+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d281a19f0 con 0x7f1d28104720 2024-05-08T05:47:11.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.992+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2806a190 con 0x7f1d28104720 2024-05-08T05:47:11.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.995+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1d1802a070 con 0x7f1d28104720 2024-05-08T05:47:11.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.995+0000 7f1d1e7fc700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1d140b2880 0x7f1d140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:11.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.995+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f1d180e2b00 con 0x7f1d28104720 2024-05-08T05:47:11.995 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.995+0000 7f1d2d76e700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1d140b2880 0x7f1d140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:11.996 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.996+0000 7f1d2d76e700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1d140b2880 0x7f1d140b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d28104580 tx=0x7f1d20009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:11.999 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:11.999+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d18031070 con 0x7f1d28104720 2024-05-08T05:47:12.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:12.211+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1d280638e0 con 0x7f1d140b2880 2024-05-08T05:47:12.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:12 smithi104 ceph-mon[34559]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:12.826 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:12 smithi104 ceph-mon[34559]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:47:12.827 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:12 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:12.827 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:12 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:12.827 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:12 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:12 smithi003 ceph-mon[30137]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:12 smithi003 ceph-mon[30137]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:47:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:12 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:12 smithi089 ceph-mon[34495]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:12 smithi089 ceph-mon[34495]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:47:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:12 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:12 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:12 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:13.788 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:13 smithi104 ceph-mon[34559]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:13.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:13 smithi003 ceph-mon[30137]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:13 smithi089 ceph-mon[34495]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:14 smithi003 ceph-mon[30137]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:14 smithi089 ceph-mon[34495]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:14 smithi104 ceph-mon[34559]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:15 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/3307311894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]: dispatch 2024-05-08T05:47:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:15 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]: dispatch 2024-05-08T05:47:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:15 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]': finished 2024-05-08T05:47:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:15 smithi003 ceph-mon[30137]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T05:47:15.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:15 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/3307311894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]: dispatch 2024-05-08T05:47:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:15 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]: dispatch 2024-05-08T05:47:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:15 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]': finished 2024-05-08T05:47:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:15 smithi089 ceph-mon[34495]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T05:47:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:15 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/3307311894' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]: dispatch 2024-05-08T05:47:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:15 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]: dispatch 2024-05-08T05:47:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:15 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4924e7d-921a-4cbc-a59a-4b23e570e1c4"}]': finished 2024-05-08T05:47:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:15 smithi104 ceph-mon[34559]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T05:47:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:16 smithi003 ceph-mon[30137]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:47:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:16 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/4049209515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:16.914 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:16 smithi089 ceph-mon[34495]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:47:16.914 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:16 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/4049209515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:16 smithi104 ceph-mon[34559]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:47:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:16 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/4049209515' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:18 smithi003 ceph-mon[30137]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T05:47:18.910 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:18 smithi104 ceph-mon[34559]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T05:47:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:18 smithi089 ceph-mon[34495]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-08T05:47:20.851 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:20 smithi104 ceph-mon[34559]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:20 smithi003 ceph-mon[30137]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:21.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:20 smithi089 ceph-mon[34495]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:22.841 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:22 smithi104 ceph-mon[34559]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T05:47:22.841 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:47:22.841 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:22 smithi003 ceph-mon[30137]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T05:47:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:47:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:23.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:22 smithi089 ceph-mon[34495]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-05-08T05:47:23.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:47:23.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:23.862 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:23 smithi104 ceph-mon[34559]: Deploying daemon osd.8 on smithi104 2024-05-08T05:47:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:23 smithi003 ceph-mon[30137]: Deploying daemon osd.8 on smithi104 2024-05-08T05:47:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:23 smithi089 ceph-mon[34495]: Deploying daemon osd.8 on smithi104 2024-05-08T05:47:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:24 smithi003 ceph-mon[30137]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:24.923 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:24 smithi104 ceph-mon[34559]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:25.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:24 smithi089 ceph-mon[34495]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:26 smithi003 ceph-mon[30137]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:27.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:26 smithi089 ceph-mon[34495]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:26 smithi104 ceph-mon[34559]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:28.748 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:28 smithi104 ceph-mon[34559]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:28.748 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:28.748 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:28.748 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:28 smithi003 ceph-mon[30137]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:29.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:28 smithi089 ceph-mon[34495]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:29.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:29.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:29.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:30 smithi003 ceph-mon[30137]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:31.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:30 smithi089 ceph-mon[34495]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:31.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:30 smithi104 ceph-mon[34559]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:32.032 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.032+0000 7f1d1e7fc700 1 -- 172.21.15.104:0/2765707819 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d280638e0 con 0x7f1d140b2880 2024-05-08T05:47:32.033 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 8 on host 'smithi104' 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1d140b2880 msgr2=0x7f1d140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1d140b2880 0x7f1d140b4d40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d28104580 tx=0x7f1d20009040 comp rx=0 tx=0).stop 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 msgr2=0x7f1d28199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28199be0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1d180036f0 tx=0x7f1d180047b0 comp rx=0 tx=0).stop 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 shutdown_connections 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1d140b2880 0x7f1d140b4d40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:32.035 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d281050c0 0x7f1d281a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d28103520 0x7f1d281996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 --2- 172.21.15.104:0/2765707819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d28104720 0x7f1d28199be0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 >> 172.21.15.104:0/2765707819 conn(0x7f1d280feac0 msgr2=0x7f1d28107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 shutdown_connections 2024-05-08T05:47:32.036 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:47:32.035+0000 7f1d2f9d2700 1 -- 172.21.15.104:0/2765707819 wait complete. 2024-05-08T05:47:32.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:47:32.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:47:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:47:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:47:32.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:32.685 DEBUG:teuthology.orchestra.run.smithi104:osd.8> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T05:47:32.687 INFO:tasks.cephadm:Deploying osd.9 on smithi003 with /dev/vg_nvme/lv_1... 2024-05-08T05:47:32.688 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:47:32.949 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:47:33.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:47:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: from='osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:47:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:47:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:47:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: from='osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:47:33.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: from='osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:47:33.330 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:33.771 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 05:47:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T05:47:33.457+0000 7feaf8c17700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T05:47:33.771 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 05:47:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T05:47:33.467+0000 7feaf0a02700 -1 osd.8 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:47:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:34 smithi003 ceph-mon[30137]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:47:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:34 smithi003 ceph-mon[30137]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:47:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:34 smithi003 ceph-mon[30137]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:34 smithi003 ceph-mon[30137]: from='osd.8 ' entity='osd.8' 2024-05-08T05:47:34.239 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:34 smithi104 ceph-mon[34559]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:47:34.240 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:34 smithi104 ceph-mon[34559]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:47:34.240 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:34.240 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:34.240 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:34 smithi104 ceph-mon[34559]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:34.240 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:34 smithi104 ceph-mon[34559]: from='osd.8 ' entity='osd.8' 2024-05-08T05:47:34.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:34 smithi089 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:47:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:34 smithi089 ceph-mon[34495]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:47:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:34 smithi089 ceph-mon[34495]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:47:34.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:34 smithi089 ceph-mon[34495]: from='osd.8 ' entity='osd.8' 2024-05-08T05:47:35.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:35 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:47:35.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:35 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:47:35.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:35 smithi003 ceph-mon[30137]: osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] boot 2024-05-08T05:47:35.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:35 smithi003 ceph-mon[30137]: osdmap e53: 9 total, 9 up, 9 in 2024-05-08T05:47:35.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:35 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:35.255 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:35 smithi003 ceph-mon[30137]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T05:47:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:47:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:47:35.255 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10+0 records in 2024-05-08T05:47:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:10+0 records out 2024-05-08T05:47:35.255 INFO:teuthology.orchestra.run.smithi003.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0120743 s, 868 MB/s 2024-05-08T05:47:35.255 INFO:teuthology.orchestra.run.smithi003.stderr:--> Zapping successful for: 2024-05-08T05:47:35.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:35 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:47:35.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:35 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:47:35.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:35 smithi104 ceph-mon[34559]: osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] boot 2024-05-08T05:47:35.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:35 smithi104 ceph-mon[34559]: osdmap e53: 9 total, 9 up, 9 in 2024-05-08T05:47:35.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:35 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:35.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:35 smithi104 ceph-mon[34559]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T05:47:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:35 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:47:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:35 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:47:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:35 smithi089 ceph-mon[34495]: osd.8 [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] boot 2024-05-08T05:47:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:35 smithi089 ceph-mon[34495]: osdmap e53: 9 total, 9 up, 9 in 2024-05-08T05:47:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:35 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:47:35.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:35 smithi089 ceph-mon[34495]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T05:47:36.165 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi003:vg_nvme/lv_1 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:36.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:36.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.425 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:47:36.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:36.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:47:36.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:36.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:36.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:47:36.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:37.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:37 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi104 2024-05-08T05:47:37.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:37 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi104 to 5696M 2024-05-08T05:47:37.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:37 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi104 2024-05-08T05:47:37.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:37 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi104 to 5696M 2024-05-08T05:47:37.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:37 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi104 2024-05-08T05:47:37.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:37 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi104 to 5696M 2024-05-08T05:47:38.297 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:38 smithi003 ceph-mon[30137]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 -- 172.21.15.3:0/1129403550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 msgr2=0x7f7b10102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 --2- 172.21.15.3:0/1129403550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b10102820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c0099c0 tx=0x7f7b0c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 -- 172.21.15.3:0/1129403550 shutdown_connections 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 --2- 172.21.15.3:0/1129403550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b10103fa0 0x7f7b1010a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 --2- 172.21.15.3:0/1129403550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b10102820 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 --2- 172.21.15.3:0/1129403550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b10103600 0x7f7b10103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 -- 172.21.15.3:0/1129403550 >> 172.21.15.3:0/1129403550 conn(0x7f7b100fd9a0 msgr2=0x7f7b100ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:47:38.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 -- 172.21.15.3:0/1129403550 shutdown_connections 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.412+0000 7f7b18463700 1 -- 172.21.15.3:0/1129403550 wait complete. 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 Processor -- start 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 -- start start 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b101985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b10103600 0x7f7b10198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b10103fa0 0x7f7b1019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010de90 con 0x7f7b10102400 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010dd10 con 0x7f7b10103fa0 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b18463700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1010e010 con 0x7f7b10103600 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b101985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b101985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35462/0 (socket says 172.21.15.3:35462) 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 -- 172.21.15.3:0/4287938720 learned_addr learned my addr 172.21.15.3:0/4287938720 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b16a00700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b10103fa0 0x7f7b1019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b159fe700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b10103600 0x7f7b10198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:38.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 -- 172.21.15.3:0/4287938720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b10103600 msgr2=0x7f7b10198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b10103600 0x7f7b10198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 -- 172.21.15.3:0/4287938720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b10103fa0 msgr2=0x7f7b1019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b10103fa0 0x7f7b1019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 -- 172.21.15.3:0/4287938720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b101a0210 con 0x7f7b10102400 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.413+0000 7f7b161ff700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b101985b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009f50 tx=0x7f7b0c005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.414+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c027070 con 0x7f7b10102400 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.414+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b0c005940 con 0x7f7b10102400 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.414+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a04a0 con 0x7f7b10102400 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.414+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c022bf0 con 0x7f7b10102400 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.414+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a0980 con 0x7f7b10102400 2024-05-08T05:47:38.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.415+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1006a190 con 0x7f7b10102400 2024-05-08T05:47:38.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.416+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7b0c02a070 con 0x7f7b10102400 2024-05-08T05:47:38.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.417+0000 7f7b077fe700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7afc0b2880 0x7f7afc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:47:38.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.417+0000 7f7b159fe700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7afc0b2880 0x7f7afc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:47:38.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.417+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5829+0+0 (secure 0 0 0) 0x7f7b0c0e2b90 con 0x7f7b10102400 2024-05-08T05:47:38.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.417+0000 7f7b159fe700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7afc0b2880 0x7f7afc0b4d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b00004630 tx=0x7f7b0000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:47:38.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.420+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b0c031070 con 0x7f7b10102400 2024-05-08T05:47:38.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:38 smithi089 ceph-mon[34495]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:38.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:38 smithi104 ceph-mon[34559]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 266 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:38.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:47:38.642+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7b100638e0 con 0x7f7afc0b2880 2024-05-08T05:47:39.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:39 smithi089 ceph-mon[34495]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:39.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:39.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:39.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:39 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:39.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:39 smithi104 ceph-mon[34559]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:39.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:39.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:39.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:39 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:39.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:39 smithi003 ceph-mon[30137]: from='client.14403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi003:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:47:39.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:47:39.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:47:39.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:39 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:40.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:40 smithi003 ceph-mon[30137]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:40.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:40 smithi089 ceph-mon[34495]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:40.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:40 smithi104 ceph-mon[34559]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:42 smithi003 ceph-mon[30137]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T05:47:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:42 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3504403066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b34df5-382a-42ff-a9ed-12c6a02706a9"}]: dispatch 2024-05-08T05:47:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:42 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3504403066' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b34df5-382a-42ff-a9ed-12c6a02706a9"}]': finished 2024-05-08T05:47:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:42 smithi003 ceph-mon[30137]: osdmap e56: 10 total, 9 up, 10 in 2024-05-08T05:47:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:47:42.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:42 smithi089 ceph-mon[34495]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T05:47:42.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:42 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3504403066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b34df5-382a-42ff-a9ed-12c6a02706a9"}]: dispatch 2024-05-08T05:47:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:42 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3504403066' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b34df5-382a-42ff-a9ed-12c6a02706a9"}]': finished 2024-05-08T05:47:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:42 smithi089 ceph-mon[34495]: osdmap e56: 10 total, 9 up, 10 in 2024-05-08T05:47:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:47:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:42 smithi104 ceph-mon[34559]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-08T05:47:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:42 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3504403066' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "99b34df5-382a-42ff-a9ed-12c6a02706a9"}]: dispatch 2024-05-08T05:47:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:42 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3504403066' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99b34df5-382a-42ff-a9ed-12c6a02706a9"}]': finished 2024-05-08T05:47:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:42 smithi104 ceph-mon[34559]: osdmap e56: 10 total, 9 up, 10 in 2024-05-08T05:47:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:47:43.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:43 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1838980661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:43.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:43 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1838980661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:43.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:43 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1838980661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:47:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:44 smithi089 ceph-mon[34495]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:44 smithi104 ceph-mon[34559]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:44.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:44 smithi003 ceph-mon[30137]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:47:46.298 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:46 smithi003 ceph-mon[30137]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:46 smithi089 ceph-mon[34495]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:46 smithi104 ceph-mon[34559]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:48.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:48 smithi003 ceph-mon[30137]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:48.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:48 smithi089 ceph-mon[34495]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:48.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:48 smithi104 ceph-mon[34559]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:50.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:50 smithi003 ceph-mon[30137]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:50.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:47:50.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:50 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:50.484 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:50 smithi003 ceph-mon[30137]: Deploying daemon osd.9 on smithi003 2024-05-08T05:47:50.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:50 smithi089 ceph-mon[34495]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:50.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:47:50.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:50 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:50.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:50 smithi089 ceph-mon[34495]: Deploying daemon osd.9 on smithi003 2024-05-08T05:47:50.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:50 smithi104 ceph-mon[34559]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:47:50.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:47:50.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:50 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:47:50.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:50 smithi104 ceph-mon[34559]: Deploying daemon osd.9 on smithi003 2024-05-08T05:47:52.343 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:52 smithi003 ceph-mon[30137]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:52.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:52 smithi089 ceph-mon[34495]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:52.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:52 smithi104 ceph-mon[34559]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:54.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:54 smithi003 ceph-mon[30137]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:54.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:54 smithi089 ceph-mon[34495]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:54.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:54 smithi104 ceph-mon[34559]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:56.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:56 smithi089 ceph-mon[34495]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:56.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:56.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:56 smithi104 ceph-mon[34559]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:56.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:56.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:56.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:56.618 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:56 smithi003 ceph-mon[30137]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:56.618 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:56.618 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:47:56.618 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:47:58.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:47:58 smithi003 ceph-mon[30137]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:58.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:47:58 smithi089 ceph-mon[34495]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:47:58.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:47:58 smithi104 ceph-mon[34559]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:00.209 INFO:teuthology.orchestra.run.smithi003.stdout:Created osd(s) 9 on host 'smithi003' 2024-05-08T05:48:00.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.208+0000 7f7b077fe700 1 -- 172.21.15.3:0/4287938720 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7b100638e0 con 0x7f7afc0b2880 2024-05-08T05:48:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7afc0b2880 msgr2=0x7f7afc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7afc0b2880 0x7f7afc0b4d40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7b00004630 tx=0x7f7b0000b3c0 comp rx=0 tx=0).stop 2024-05-08T05:48:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 msgr2=0x7f7b101985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b101985b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c009f50 tx=0x7f7b0c005af0 comp rx=0 tx=0).stop 2024-05-08T05:48:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 shutdown_connections 2024-05-08T05:48:00.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b10103fa0 0x7f7b1019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7afc0b2880 0x7f7afc0b4d40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b10102400 0x7f7b101985b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 --2- 172.21.15.3:0/4287938720 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b10103600 0x7f7b10198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.211+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 >> 172.21.15.3:0/4287938720 conn(0x7f7b100fd9a0 msgr2=0x7f7b101069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.212+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 shutdown_connections 2024-05-08T05:48:00.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:00.212+0000 7f7b18463700 1 -- 172.21.15.3:0/4287938720 wait complete. 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:00.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:00 smithi003 ceph-mon[30137]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:48:00.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:00.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:00.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:00.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:00 smithi089 ceph-mon[34495]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:48:00.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:00.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:00.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:00.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:00.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:00 smithi104 ceph-mon[34559]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:48:01.033 DEBUG:teuthology.orchestra.run.smithi003:osd.9> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T05:48:01.035 INFO:tasks.cephadm:Deploying osd.10 on smithi089 with /dev/vg_nvme/lv_1... 2024-05-08T05:48:01.036 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:48:01.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:48:01.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T05:48:01.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:48:01.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:01.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:01.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:01.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:01 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:01.270 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 05:48:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T05:48:01.134+0000 7fba52f39700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T05:48:01.270 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 05:48:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T05:48:01.146+0000 7fba4ed2c700 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:48:01.283 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:01.405 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:01 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]: dispatch 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:01.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:01 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:02.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:02 smithi003 ceph-mon[30137]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:48:02.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:02 smithi003 ceph-mon[30137]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:48:02.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:02 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:02 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:02 smithi003 ceph-mon[30137]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:02.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:02 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:02 smithi089 ceph-mon[34495]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:48:02.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:02 smithi089 ceph-mon[34495]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:48:02.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:02 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:02 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:02 smithi089 ceph-mon[34495]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:02.504 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:02 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:02 smithi104 ceph-mon[34559]: from='osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi003", "root=default"]}]': finished 2024-05-08T05:48:02.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:02 smithi104 ceph-mon[34559]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:48:02.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:02 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:02 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:02.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:02 smithi104 ceph-mon[34559]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 267 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:48:02.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:02 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:48:03.401 INFO:teuthology.orchestra.run.smithi089.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:48:03.401 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10+0 records in 2024-05-08T05:48:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:10+0 records out 2024-05-08T05:48:03.402 INFO:teuthology.orchestra.run.smithi089.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122896 s, 853 MB/s 2024-05-08T05:48:03.402 INFO:teuthology.orchestra.run.smithi089.stderr:--> Zapping successful for: 2024-05-08T05:48:03.402 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:03 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:48:03.402 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:03 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:48:03.402 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:03 smithi089 ceph-mon[34495]: osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] boot 2024-05-08T05:48:03.402 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:03 smithi089 ceph-mon[34495]: osdmap e59: 10 total, 10 up, 10 in 2024-05-08T05:48:03.402 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:03 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:03.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:03 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:48:03.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:03 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:48:03.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:03 smithi003 ceph-mon[30137]: osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] boot 2024-05-08T05:48:03.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:03 smithi003 ceph-mon[30137]: osdmap e59: 10 total, 10 up, 10 in 2024-05-08T05:48:03.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:03 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:03.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:03 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:48:03.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:03 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:48:03.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:03 smithi104 ceph-mon[34559]: osd.9 [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] boot 2024-05-08T05:48:03.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:03 smithi104 ceph-mon[34559]: osdmap e59: 10 total, 10 up, 10 in 2024-05-08T05:48:03.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:03 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:48:04.030 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi089:vg_nvme/lv_1 2024-05-08T05:48:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:04 smithi089 ceph-mon[34495]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T05:48:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:04 smithi089 ceph-mon[34495]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:04.272 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:48:04.356 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:04 smithi003 ceph-mon[30137]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T05:48:04.356 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:04 smithi003 ceph-mon[30137]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:04 smithi104 ceph-mon[34559]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T05:48:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:04 smithi104 ceph-mon[34559]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:05.673 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi003 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi003 to 4273M 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:05.674 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:05 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi003 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi003 to 4273M 2024-05-08T05:48:05.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:05.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:05.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:05 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2242407941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 msgr2=0x7fb0e8102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:05.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2242407941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e8102840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb0d80099c0 tx=0x7fb0d8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:48:05.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2242407941 shutdown_connections 2024-05-08T05:48:05.893 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2242407941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e8103fc0 0x7fb0e810a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2242407941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e8102840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2242407941 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103620 0x7fb0e8103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2242407941 >> 172.21.15.89:0/2242407941 conn(0x7fb0e80fda00 msgr2=0x7fb0e80ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2242407941 shutdown_connections 2024-05-08T05:48:05.894 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2242407941 wait complete. 2024-05-08T05:48:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 Processor -- start 2024-05-08T05:48:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.893+0000 7fb0ec9c0700 1 -- start start 2024-05-08T05:48:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0ec9c0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e81063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0ec9c0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e8103620 0x7fb0e8104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0ec9c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103fc0 0x7fb0e8104f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:05.896 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0ec9c0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810df10 con 0x7fb0e8103fc0 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0ec9c0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810dd90 con 0x7fb0e8102420 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0ec9c0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e810e090 con 0x7fb0e8103620 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e81063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e81063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:50760/0 (socket says 172.21.15.89:50760) 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 -- 172.21.15.89:0/2616438526 learned_addr learned my addr 172.21.15.89:0/2616438526 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e5d9b700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e8103620 0x7fb0e8104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e6d9d700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103fc0 0x7fb0e8104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 -- 172.21.15.89:0/2616438526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e8103620 msgr2=0x7fb0e8104a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e8103620 0x7fb0e8104a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 -- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103fc0 msgr2=0x7fb0e8104f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103fc0 0x7fb0e8104f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 -- 172.21.15.89:0/2616438526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e81057f0 con 0x7fb0e8102420 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e6d9d700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103fc0 0x7fb0e8104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0e659c700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e81063b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009f50 tx=0x7fb0d80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:05.897 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.894+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8027070 con 0x7fb0e8102420 2024-05-08T05:48:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.895+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e81a3770 con 0x7fb0e8102420 2024-05-08T05:48:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.895+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0d8022470 con 0x7fb0e8102420 2024-05-08T05:48:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.895+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d800f6c0 con 0x7fb0e8102420 2024-05-08T05:48:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.895+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a3c50 con 0x7fb0e8102420 2024-05-08T05:48:05.898 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.895+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e806a190 con 0x7fb0e8102420 2024-05-08T05:48:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.897+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb0d802a070 con 0x7fb0e8102420 2024-05-08T05:48:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.897+0000 7fb0d77fe700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb0d00b2880 0x7fb0d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.898+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6261+0+0 (secure 0 0 0) 0x7fb0d80e2d80 con 0x7fb0e8102420 2024-05-08T05:48:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.898+0000 7fb0e5d9b700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb0d00b2880 0x7fb0d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:05.899 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.898+0000 7fb0e5d9b700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb0d00b2880 0x7fb0d00b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00ac50 tx=0x7fb0dc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:05.901 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:05.901+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0d8013070 con 0x7fb0e8102420 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi003 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi003 to 4273M 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:05 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:06.116 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:06.116+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb0e80ffba0 con 0x7fb0d00b2880 2024-05-08T05:48:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:06 smithi089 ceph-mon[34495]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:06 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:06 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:06.728 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:06 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:06 smithi104 ceph-mon[34559]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:06 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:06 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:06.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:06 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:06 smithi003 ceph-mon[30137]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:06 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:07.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:07 smithi089 ceph-mon[34495]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:07.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:07 smithi104 ceph-mon[34559]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:07.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:07 smithi003 ceph-mon[30137]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi089:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:08.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:08 smithi089 ceph-mon[34495]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:08.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:08 smithi104 ceph-mon[34559]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:08.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:08 smithi003 ceph-mon[30137]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:09.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:09 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/3672985367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]: dispatch 2024-05-08T05:48:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:09 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]: dispatch 2024-05-08T05:48:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:09 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]': finished 2024-05-08T05:48:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:09 smithi089 ceph-mon[34495]: osdmap e61: 11 total, 10 up, 11 in 2024-05-08T05:48:09.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:09 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:09.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:09 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/3672985367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]: dispatch 2024-05-08T05:48:09.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:09 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]: dispatch 2024-05-08T05:48:09.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:09 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]': finished 2024-05-08T05:48:09.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:09 smithi104 ceph-mon[34559]: osdmap e61: 11 total, 10 up, 11 in 2024-05-08T05:48:09.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:09 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:09.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:09 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/3672985367' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]: dispatch 2024-05-08T05:48:09.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:09 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]: dispatch 2024-05-08T05:48:09.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:09 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e939df5-46fd-4a17-ba2d-e3f72120e35e"}]': finished 2024-05-08T05:48:09.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:09 smithi003 ceph-mon[30137]: osdmap e61: 11 total, 10 up, 11 in 2024-05-08T05:48:09.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:09 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:10.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:10 smithi089 ceph-mon[34495]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:10 smithi089 ceph-mon[34495]: from='client.? 172.21.15.89:0/838573867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:10 smithi104 ceph-mon[34559]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:10 smithi104 ceph-mon[34559]: from='client.? 172.21.15.89:0/838573867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:10 smithi003 ceph-mon[30137]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:10.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:10 smithi003 ceph-mon[30137]: from='client.? 172.21.15.89:0/838573867' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:12.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:12 smithi089 ceph-mon[34495]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:12.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:12 smithi104 ceph-mon[34559]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:12.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:12 smithi003 ceph-mon[30137]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:14.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:14 smithi089 ceph-mon[34495]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:14.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:14 smithi104 ceph-mon[34559]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:14 smithi003 ceph-mon[30137]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:16.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:16 smithi089 ceph-mon[34495]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:16.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:16 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:48:16.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:16 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:16.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:16 smithi104 ceph-mon[34559]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:16.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:16 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:48:16.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:16 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:16 smithi003 ceph-mon[30137]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:16 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:48:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:16 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:17.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:17 smithi089 ceph-mon[34495]: Deploying daemon osd.10 on smithi089 2024-05-08T05:48:17.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:17 smithi104 ceph-mon[34559]: Deploying daemon osd.10 on smithi089 2024-05-08T05:48:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:17 smithi003 ceph-mon[30137]: Deploying daemon osd.10 on smithi089 2024-05-08T05:48:18.629 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:18 smithi089 ceph-mon[34495]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:18.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:18 smithi104 ceph-mon[34559]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:18.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:18 smithi003 ceph-mon[30137]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:20.731 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:20 smithi089 ceph-mon[34495]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:20 smithi104 ceph-mon[34559]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:20 smithi003 ceph-mon[30137]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:22.640 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:22 smithi089 ceph-mon[34495]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:22.641 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:22.641 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:22.641 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:22 smithi104 ceph-mon[34559]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:22 smithi003 ceph-mon[30137]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:22.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:24.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:24 smithi089 ceph-mon[34495]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:24.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:24 smithi104 ceph-mon[34559]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:24 smithi003 ceph-mon[30137]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:25.491 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.490+0000 7fb0d77fe700 1 -- 172.21.15.89:0/2616438526 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb0e80ffba0 con 0x7fb0d00b2880 2024-05-08T05:48:25.491 INFO:teuthology.orchestra.run.smithi089.stdout:Created osd(s) 10 on host 'smithi089' 2024-05-08T05:48:25.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb0d00b2880 msgr2=0x7fb0d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb0d00b2880 0x7fb0d00b4d40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00ac50 tx=0x7fb0dc00a380 comp rx=0 tx=0).stop 2024-05-08T05:48:25.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 msgr2=0x7fb0e81063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:25.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e81063b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009f50 tx=0x7fb0d80047b0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 shutdown_connections 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb0d00b2880 0x7fb0d00b4d40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb0e8103fc0 0x7fb0e8104f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb0e8102420 0x7fb0e81063b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 --2- 172.21.15.89:0/2616438526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0e8103620 0x7fb0e8104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 >> 172.21.15.89:0/2616438526 conn(0x7fb0e80fda00 msgr2=0x7fb0e80ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 shutdown_connections 2024-05-08T05:48:25.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-05-08T05:48:25.493+0000 7fb0ec9c0700 1 -- 172.21.15.89:0/2616438526 wait complete. 2024-05-08T05:48:26.093 DEBUG:teuthology.orchestra.run.smithi089:osd.10> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T05:48:26.096 INFO:tasks.cephadm:Deploying osd.11 on smithi104 with /dev/vg_nvme/lv_1... 2024-05-08T05:48:26.096 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:48:26.341 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:48:26.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:26.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: from='osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:48:26.712 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:48:26.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:26.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: from='osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:48:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:48:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: from='osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:26.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:27.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:27 smithi089 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:48:27.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:27 smithi089 ceph-mon[34495]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T05:48:27.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:27 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:27.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:27 smithi089 ceph-mon[34495]: from='osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:48:27.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:27 smithi089 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:48:27.763 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 05:48:27 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T05:48:27.489+0000 7f4a15c45700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T05:48:27.763 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 05:48:27 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T05:48:27.498+0000 7f4a11a38700 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:48:27.767 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:27 smithi104 ceph-mon[34559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:48:27.767 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:27 smithi104 ceph-mon[34559]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T05:48:27.767 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:27 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:27.767 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:27 smithi104 ceph-mon[34559]: from='osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:48:27.767 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:27 smithi104 ceph-mon[34559]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:48:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:27 smithi003 ceph-mon[30137]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:48:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:27 smithi003 ceph-mon[30137]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T05:48:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:27 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:27 smithi003 ceph-mon[30137]: from='osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:48:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:27 smithi003 ceph-mon[30137]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]: dispatch 2024-05-08T05:48:28.522 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:48:28.522 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:48:28.522 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-05-08T05:48:28.522 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-05-08T05:48:28.522 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0141851 s, 739 MB/s 2024-05-08T05:48:28.522 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-05-08T05:48:28.551 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:28 smithi104 ceph-mon[34559]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:28.551 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:28 smithi104 ceph-mon[34559]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:48:28.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:28 smithi104 ceph-mon[34559]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:48:28.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:28.552 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:28.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:28 smithi089 ceph-mon[34495]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:28.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:28 smithi089 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:48:28.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:28 smithi089 ceph-mon[34495]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:48:28.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:28.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:28 smithi003 ceph-mon[30137]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:48:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:28 smithi003 ceph-mon[30137]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi089", "root=default"]}]': finished 2024-05-08T05:48:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:28 smithi003 ceph-mon[30137]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:48:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:29.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-05-08T05:48:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:29 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:48:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:29 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:48:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:29 smithi104 ceph-mon[34559]: osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] boot 2024-05-08T05:48:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:29 smithi104 ceph-mon[34559]: osdmap e64: 11 total, 11 up, 11 in 2024-05-08T05:48:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:29 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:29 smithi104 ceph-mon[34559]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T05:48:29.601 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:48:29.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:29 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:48:29.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:29 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:48:29.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:29 smithi089 ceph-mon[34495]: osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] boot 2024-05-08T05:48:29.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:29 smithi089 ceph-mon[34495]: osdmap e64: 11 total, 11 up, 11 in 2024-05-08T05:48:29.650 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:29 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:29.651 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:29 smithi089 ceph-mon[34495]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T05:48:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:29 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:48:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:29 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:48:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:29 smithi003 ceph-mon[30137]: osd.10 [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] boot 2024-05-08T05:48:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:29 smithi003 ceph-mon[30137]: osdmap e64: 11 total, 11 up, 11 in 2024-05-08T05:48:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:29 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:48:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:29 smithi003 ceph-mon[30137]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T05:48:30.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:30.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:30 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:30 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:30.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:30 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:31.365 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 -- 172.21.15.104:0/281534050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 msgr2=0x7f73f8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 --2- 172.21.15.104:0/281534050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8102820 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f73e8009990 tx=0x7f73e8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 -- 172.21.15.104:0/281534050 shutdown_connections 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 --2- 172.21.15.104:0/281534050 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f8103fa0 0x7f73f810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 --2- 172.21.15.104:0/281534050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8102820 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 --2- 172.21.15.104:0/281534050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f8103600 0x7f73f8103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.363+0000 7f73fe233700 1 -- 172.21.15.104:0/281534050 >> 172.21.15.104:0/281534050 conn(0x7f73f80fd9a0 msgr2=0x7f73f80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 -- 172.21.15.104:0/281534050 shutdown_connections 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 -- 172.21.15.104:0/281534050 wait complete. 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 Processor -- start 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 -- start start 2024-05-08T05:48:31.366 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f8103600 0x7f73f8199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f8103fa0 0x7f73f81a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810de90 con 0x7f73f8103fa0 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810dd10 con 0x7f73f8103600 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.364+0000 7f73fe233700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f73f810e010 con 0x7f73f8102400 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57518/0 (socket says 172.21.15.104:57518) 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 -- 172.21.15.104:0/4253409374 learned_addr learned my addr 172.21.15.104:0/4253409374 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f6ffd700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f8103600 0x7f73f8199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f7fff700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f8103fa0 0x7f73f81a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 -- 172.21.15.104:0/4253409374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f8103600 msgr2=0x7f73f8199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f8103600 0x7f73f8199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 -- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f8103fa0 msgr2=0x7f73f81a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:31.367 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f8103fa0 0x7f73f81a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 -- 172.21.15.104:0/4253409374 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f81a12d0 con 0x7f73f8102400 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f77fe700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8199670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73e80059d0 tx=0x7f73e80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8027070 con 0x7f73f8102400 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f81a1560 con 0x7f73f8102400 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73e80041c0 con 0x7f73f8102400 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e800f460 con 0x7f73f8102400 2024-05-08T05:48:31.368 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.365+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a1a10 con 0x7f73f8102400 2024-05-08T05:48:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.366+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73f806a190 con 0x7f73f8102400 2024-05-08T05:48:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.368+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f73e802a070 con 0x7f73f8102400 2024-05-08T05:48:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.369+0000 7f73f4ff9700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f73e00b2880 0x7f73e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.369+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6694+0+0 (secure 0 0 0) 0x7f73e80e2d70 con 0x7f73f8102400 2024-05-08T05:48:31.369 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.369+0000 7f73f6ffd700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f73e00b2880 0x7f73e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:31.370 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.370+0000 7f73f6ffd700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f73e00b2880 0x7f73e00b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73ec005950 tx=0x7f73ec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:31.373 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.373+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73e8013070 con 0x7f73f8102400 2024-05-08T05:48:31.591 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:31.591+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f73f81a1d30 con 0x7f73e00b2880 2024-05-08T05:48:31.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:31 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi089 2024-05-08T05:48:31.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:31 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi089 to 4272M 2024-05-08T05:48:31.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:31 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi089 2024-05-08T05:48:31.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:31 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi089 to 4272M 2024-05-08T05:48:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:31 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi089 2024-05-08T05:48:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:31 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi089 to 4272M 2024-05-08T05:48:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:32 smithi089 ceph-mon[34495]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:32 smithi089 ceph-mon[34495]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:32.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:32 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:32 smithi104 ceph-mon[34559]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:32 smithi104 ceph-mon[34559]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:32 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:32 smithi003 ceph-mon[30137]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:32 smithi003 ceph-mon[30137]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:48:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:48:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:48:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:32 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:34 smithi089 ceph-mon[34495]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:34 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/3480681373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]: dispatch 2024-05-08T05:48:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:34 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]: dispatch 2024-05-08T05:48:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:34 smithi089 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]': finished 2024-05-08T05:48:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:34 smithi089 ceph-mon[34495]: osdmap e67: 12 total, 11 up, 12 in 2024-05-08T05:48:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:34 smithi104 ceph-mon[34559]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:34 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/3480681373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]: dispatch 2024-05-08T05:48:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:34 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]: dispatch 2024-05-08T05:48:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:34 smithi104 ceph-mon[34559]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]': finished 2024-05-08T05:48:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:34 smithi104 ceph-mon[34559]: osdmap e67: 12 total, 11 up, 12 in 2024-05-08T05:48:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:34 smithi003 ceph-mon[30137]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:34 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/3480681373' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]: dispatch 2024-05-08T05:48:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:34 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]: dispatch 2024-05-08T05:48:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:34 smithi003 ceph-mon[30137]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "262280c4-ebf6-4ea0-ba58-3c2eeab79323"}]': finished 2024-05-08T05:48:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:34 smithi003 ceph-mon[30137]: osdmap e67: 12 total, 11 up, 12 in 2024-05-08T05:48:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:35 smithi104 ceph-mon[34559]: from='client.? 172.21.15.104:0/3471606676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:35.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:35 smithi003 ceph-mon[30137]: from='client.? 172.21.15.104:0/3471606676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:35 smithi089 ceph-mon[34495]: from='client.? 172.21.15.104:0/3471606676' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:48:36.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:36 smithi089 ceph-mon[34495]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:48:36.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:36 smithi104 ceph-mon[34559]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:48:36.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:36 smithi003 ceph-mon[30137]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:48:38.739 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:38 smithi104 ceph-mon[34559]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:48:38.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:38 smithi003 ceph-mon[30137]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:48:39.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:38 smithi089 ceph-mon[34495]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:48:40.748 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:40 smithi104 ceph-mon[34559]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T05:48:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:40 smithi003 ceph-mon[30137]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T05:48:41.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:40 smithi089 ceph-mon[34495]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-05-08T05:48:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:42 smithi104 ceph-mon[34559]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:48:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:42.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:42 smithi003 ceph-mon[30137]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:42.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:48:42.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:43.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:42 smithi089 ceph-mon[34495]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:43.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:48:43.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:43.780 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:43 smithi104 ceph-mon[34559]: Deploying daemon osd.11 on smithi104 2024-05-08T05:48:43.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:43 smithi003 ceph-mon[30137]: Deploying daemon osd.11 on smithi104 2024-05-08T05:48:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:43 smithi089 ceph-mon[34495]: Deploying daemon osd.11 on smithi104 2024-05-08T05:48:44.639 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:44 smithi104 ceph-mon[34559]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:44 smithi003 ceph-mon[30137]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:45.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:44 smithi089 ceph-mon[34495]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:48:46.793 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:46 smithi104 ceph-mon[34559]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:46.815 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:46 smithi089 ceph-mon[34495]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:46.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:46 smithi003 ceph-mon[30137]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:48.685 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:48 smithi104 ceph-mon[34559]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:48.686 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:48.686 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:48.686 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:48 smithi003 ceph-mon[30137]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:48.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:49.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:48 smithi089 ceph-mon[34495]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:49.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:49.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:49.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:50.618 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:50 smithi104 ceph-mon[34559]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:50 smithi003 ceph-mon[30137]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:51.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:50 smithi089 ceph-mon[34495]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:51.564 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.563+0000 7f73f4ff9700 1 -- 172.21.15.104:0/4253409374 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f73f81a1d30 con 0x7f73e00b2880 2024-05-08T05:48:51.564 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 11 on host 'smithi104' 2024-05-08T05:48:51.566 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f73e00b2880 msgr2=0x7f73e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:51.566 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f73e00b2880 0x7f73e00b4d40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73ec005950 tx=0x7f73ec0058e0 comp rx=0 tx=0).stop 2024-05-08T05:48:51.566 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 msgr2=0x7f73f8199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8199670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f73e80059d0 tx=0x7f73e80047b0 comp rx=0 tx=0).stop 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 shutdown_connections 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f73e00b2880 0x7f73e00b4d40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f73f8103fa0 0x7f73f81a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f73f8102400 0x7f73f8199670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 --2- 172.21.15.104:0/4253409374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f73f8103600 0x7f73f8199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 >> 172.21.15.104:0/4253409374 conn(0x7f73f80fd9a0 msgr2=0x7f73f806b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 shutdown_connections 2024-05-08T05:48:51.567 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:48:51.566+0000 7f73fe233700 1 -- 172.21.15.104:0/4253409374 wait complete. 2024-05-08T05:48:52.307 DEBUG:teuthology.orchestra.run.smithi104:osd.11> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T05:48:52.309 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T05:48:52.309 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T05:48:52.577 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:48:52.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:52.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:52.596 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:52.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: from='osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:48:52.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:48:52.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:52.597 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: from='osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: from='osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:48:53.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:48:53.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:53.834 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:53 smithi003 ceph-mon[30137]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:48:53.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:53 smithi003 ceph-mon[30137]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T05:48:53.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:53 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:53.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:53 smithi003 ceph-mon[30137]: from='osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:48:53.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:53 smithi003 ceph-mon[30137]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:48:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:53 smithi089 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:48:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:53 smithi089 ceph-mon[34495]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T05:48:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:53 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:53 smithi089 ceph-mon[34495]: from='osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:48:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:53 smithi089 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:48:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:53 smithi104 ceph-mon[34559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:48:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:53 smithi104 ceph-mon[34559]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T05:48:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:53 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:53 smithi104 ceph-mon[34559]: from='osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:48:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:53 smithi104 ceph-mon[34559]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-05-08T05:48:54.022 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 05:48:53 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T05:48:53.567+0000 7fdc3ebfa700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T05:48:54.022 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 05:48:53 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T05:48:53.585+0000 7fdc3a9ed700 -1 osd.11 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:48:54.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 -- 172.21.15.3:0/2877810799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00102420 msgr2=0x7f1e00102840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 --2- 172.21.15.3:0/2877810799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00102420 0x7f1e00102840 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1df00099c0 tx=0x7f1df0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 -- 172.21.15.3:0/2877810799 shutdown_connections 2024-05-08T05:48:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 --2- 172.21.15.3:0/2877810799 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00103fc0 0x7f1e0010a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 --2- 172.21.15.3:0/2877810799 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00102420 0x7f1e00102840 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 --2- 172.21.15.3:0/2877810799 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00103620 0x7f1e00103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 -- 172.21.15.3:0/2877810799 >> 172.21.15.3:0/2877810799 conn(0x7f1e000fd9c0 msgr2=0x7f1e000ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 -- 172.21.15.3:0/2877810799 shutdown_connections 2024-05-08T05:48:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.398+0000 7f1e063e8700 1 -- 172.21.15.3:0/2877810799 wait complete. 2024-05-08T05:48:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 Processor -- start 2024-05-08T05:48:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 -- start start 2024-05-08T05:48:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00102420 0x7f1e001063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00103620 0x7f1e00104a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 0x7f1e00104f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0010dec0 con 0x7f1e00103fc0 2024-05-08T05:48:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0010dd40 con 0x7f1e00102420 2024-05-08T05:48:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e063e8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0010e040 con 0x7f1e00103620 2024-05-08T05:48:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 0x7f1e00104f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 0x7f1e00104f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59114/0 (socket says 172.21.15.3:59114) 2024-05-08T05:48:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 -- 172.21.15.3:0/3662541421 learned_addr learned my addr 172.21.15.3:0/3662541421 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:48:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1dff7fe700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00103620 0x7f1e00104a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:54.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1dfffff700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00102420 0x7f1e001063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 -- 172.21.15.3:0/3662541421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00103620 msgr2=0x7f1e00104a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00103620 0x7f1e00104a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 -- 172.21.15.3:0/3662541421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00102420 msgr2=0x7f1e001063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00102420 0x7f1e001063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.399+0000 7f1e04985700 1 -- 172.21.15.3:0/3662541421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e001057f0 con 0x7f1e00103fc0 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.400+0000 7f1e04985700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 0x7f1e00104f40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1df800ef30 tx=0x7f1df800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:54.407 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.400+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80092e0 con 0x7f1e00103fc0 2024-05-08T05:48:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.400+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df8004bb0 con 0x7f1e00103fc0 2024-05-08T05:48:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.400+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e001a3750 con 0x7f1e00103fc0 2024-05-08T05:48:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.400+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8004280 con 0x7f1e00103fc0 2024-05-08T05:48:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.400+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e001a3c90 con 0x7f1e00103fc0 2024-05-08T05:48:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.401+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e0006a190 con 0x7f1e00103fc0 2024-05-08T05:48:54.408 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.402+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f1df801b030 con 0x7f1e00103fc0 2024-05-08T05:48:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.403+0000 7f1dfd7fa700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1de80b2900 0x7f1de80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.403+0000 7f1dfffff700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1de80b2900 0x7f1de80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.403+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6847+0+0 (secure 0 0 0) 0x7f1df80e2220 con 0x7f1e00103fc0 2024-05-08T05:48:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.404+0000 7f1dfffff700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1de80b2900 0x7f1de80b4dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b580 tx=0x7f1df0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:54.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.407+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1df8022020 con 0x7f1e00103fc0 2024-05-08T05:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.626+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1e001a4170 con 0x7f1e00103fc0 2024-05-08T05:48:54.626 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.626+0000 7f1dfd7fa700 1 -- 172.21.15.3:0/3662541421 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f1df8010040 con 0x7f1e00103fc0 2024-05-08T05:48:54.627 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.628+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1de80b2900 msgr2=0x7f1de80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.628+0000 7f1e063e8700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1de80b2900 0x7f1de80b4dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1df000b580 tx=0x7f1df0005fb0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.628+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 msgr2=0x7f1e00104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.628+0000 7f1e063e8700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 0x7f1e00104f40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1df800ef30 tx=0x7f1df800c450 comp rx=0 tx=0).stop 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 shutdown_connections 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f1de80b2900 0x7f1de80b4dc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1e00103fc0 0x7f1e00104f40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1e00102420 0x7f1e001063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 --2- 172.21.15.3:0/3662541421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1e00103620 0x7f1e00104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:54.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 >> 172.21.15.3:0/3662541421 conn(0x7f1e000fd9c0 msgr2=0x7f1e000ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:54.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 shutdown_connections 2024-05-08T05:48:54.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:54.629+0000 7f1e063e8700 1 -- 172.21.15.3:0/3662541421 wait complete. 2024-05-08T05:48:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:54 smithi104 ceph-mon[34559]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:54 smithi104 ceph-mon[34559]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:48:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:54 smithi104 ceph-mon[34559]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:48:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:54 smithi003 ceph-mon[30137]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:54 smithi003 ceph-mon[30137]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:48:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:54 smithi003 ceph-mon[30137]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:48:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:55.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:54 smithi089 ceph-mon[34495]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:48:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:54 smithi089 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-05-08T05:48:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:54 smithi089 ceph-mon[34495]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:48:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:55.331 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1715147334,"num_in_osds":12,"osd_in_since":1715147314,"num_remapped_pgs":0} 2024-05-08T05:48:55.332 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:48:55.595 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:48:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:55 smithi104 ceph-mon[34559]: purged_snaps scrub starts 2024-05-08T05:48:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:55 smithi104 ceph-mon[34559]: purged_snaps scrub ok 2024-05-08T05:48:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:55 smithi104 ceph-mon[34559]: osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] boot 2024-05-08T05:48:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:55 smithi104 ceph-mon[34559]: osdmap e70: 12 total, 12 up, 12 in 2024-05-08T05:48:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:55 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:55.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:55 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3662541421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:48:55.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:55 smithi003 ceph-mon[30137]: purged_snaps scrub starts 2024-05-08T05:48:55.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:55 smithi003 ceph-mon[30137]: purged_snaps scrub ok 2024-05-08T05:48:55.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:55 smithi003 ceph-mon[30137]: osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] boot 2024-05-08T05:48:55.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:55 smithi003 ceph-mon[30137]: osdmap e70: 12 total, 12 up, 12 in 2024-05-08T05:48:55.850 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:55 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:55.850 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:55 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3662541421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:48:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:55 smithi089 ceph-mon[34495]: purged_snaps scrub starts 2024-05-08T05:48:56.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:55 smithi089 ceph-mon[34495]: purged_snaps scrub ok 2024-05-08T05:48:56.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:55 smithi089 ceph-mon[34495]: osd.11 [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] boot 2024-05-08T05:48:56.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:55 smithi089 ceph-mon[34495]: osdmap e70: 12 total, 12 up, 12 in 2024-05-08T05:48:56.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:55 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:48:56.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:55 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3662541421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:48:56.848 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:48:56.848 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: osdmap e71: 12 total, 12 up, 12 in 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:56.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:56.850 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: osdmap e71: 12 total, 12 up, 12 in 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:56.866 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:56.867 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:56.867 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:56.867 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:56.867 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:56.867 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: osdmap e71: 12 total, 12 up, 12 in 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:57.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:57.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:48:57.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:48:57.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:48:57.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 -- 172.21.15.3:0/4010847813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0104100 msgr2=0x7f45e0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 --2- 172.21.15.3:0/4010847813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0104100 0x7f45e0108120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f45d40099c0 tx=0x7f45d4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 -- 172.21.15.3:0/4010847813 shutdown_connections 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 --2- 172.21.15.3:0/4010847813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45e0108850 0x7f45e01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 --2- 172.21.15.3:0/4010847813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0103750 0x7f45e0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 --2- 172.21.15.3:0/4010847813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0104100 0x7f45e0108120 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 -- 172.21.15.3:0/4010847813 >> 172.21.15.3:0/4010847813 conn(0x7f45e007a600 msgr2=0x7f45e007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 -- 172.21.15.3:0/4010847813 shutdown_connections 2024-05-08T05:48:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.208+0000 7f45e6020700 1 -- 172.21.15.3:0/4010847813 wait complete. 2024-05-08T05:48:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 Processor -- start 2024-05-08T05:48:57.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 -- start start 2024-05-08T05:48:57.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 0x7f45e00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0104100 0x7f45e0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45e0108850 0x7f45e0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0113610 con 0x7f45e0103750 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0113490 con 0x7f45e0108850 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45e6020700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0113790 con 0x7f45e0104100 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45df7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 0x7f45e00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45df7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 0x7f45e00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59134/0 (socket says 172.21.15.3:59134) 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45df7fe700 1 -- 172.21.15.3:0/3614278442 learned_addr learned my addr 172.21.15.3:0/3614278442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45dffff700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45e0108850 0x7f45e0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45deffd700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0104100 0x7f45e0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45df7fe700 1 -- 172.21.15.3:0/3614278442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0104100 msgr2=0x7f45e0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.209+0000 7f45df7fe700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0104100 0x7f45e0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45df7fe700 1 -- 172.21.15.3:0/3614278442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45e0108850 msgr2=0x7f45e0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45df7fe700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45e0108850 0x7f45e0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45df7fe700 1 -- 172.21.15.3:0/3614278442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e01a8090 con 0x7f45e0103750 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45deffd700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0104100 0x7f45e0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45df7fe700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 0x7f45e00757a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45d000d950 tx=0x7f45d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0016070 con 0x7f45e0103750 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e01a8380 con 0x7f45e0103750 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45d0009df0 con 0x7f45e0103750 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d000f3c0 con 0x7f45e0103750 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.210+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e01a8890 con 0x7f45e0103750 2024-05-08T05:48:57.213 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.211+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45e006a190 con 0x7f45e0103750 2024-05-08T05:48:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.212+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f45d0019070 con 0x7f45e0103750 2024-05-08T05:48:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.213+0000 7f45dcff9700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f45c80b2880 0x7f45c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:48:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.213+0000 7f45deffd700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f45c80b2880 0x7f45c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:48:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.213+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f45d00e1990 con 0x7f45e0103750 2024-05-08T05:48:57.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.214+0000 7f45deffd700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f45c80b2880 0x7f45c80b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f45e0077230 tx=0x7f45d4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:48:57.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.218+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45d00a9b50 con 0x7f45e0103750 2024-05-08T05:48:57.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.440+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45e0076cd0 con 0x7f45e0103750 2024-05-08T05:48:57.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.441+0000 7f45dcff9700 1 -- 172.21.15.3:0/3614278442 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+18373 (secure 0 0 0) 0x7f45d00a9450 con 0x7f45e0103750 2024-05-08T05:48:57.441 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:48:57.442 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":71,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:48:55.570111+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.443+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f45c80b2880 msgr2=0x7f45c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.443+0000 7f45e6020700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f45c80b2880 0x7f45c80b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f45e0077230 tx=0x7f45d4005fd0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.443+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 msgr2=0x7f45e00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.443+0000 7f45e6020700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 0x7f45e00757a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45d000d950 tx=0x7f45d000dc60 comp rx=0 tx=0).stop 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 shutdown_connections 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f45e0108850 0x7f45e0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f45c80b2880 0x7f45c80b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f45e0103750 0x7f45e00757a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 --2- 172.21.15.3:0/3614278442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e0104100 0x7f45e0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:48:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 >> 172.21.15.3:0/3614278442 conn(0x7f45e007a600 msgr2=0x7f45e0101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:48:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 shutdown_connections 2024-05-08T05:48:57.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:48:57.444+0000 7f45e6020700 1 -- 172.21.15.3:0/3614278442 wait complete. 2024-05-08T05:48:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:57 smithi003 ceph-mon[30137]: Detected new or changed devices on smithi104 2024-05-08T05:48:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:57 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi104 to 4272M 2024-05-08T05:48:57.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:57 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3614278442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:48:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:57 smithi089 ceph-mon[34495]: Detected new or changed devices on smithi104 2024-05-08T05:48:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:57 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi104 to 4272M 2024-05-08T05:48:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:57 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3614278442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:48:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:57 smithi104 ceph-mon[34559]: Detected new or changed devices on smithi104 2024-05-08T05:48:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:57 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi104 to 4272M 2024-05-08T05:48:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:57 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3614278442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:48:58.235 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:45:01.214856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:48:58.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T05:48:58.484 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:48:58.850 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:48:58 smithi003 ceph-mon[30137]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:48:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:48:58 smithi089 ceph-mon[34495]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:48:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:48:58 smithi104 ceph-mon[34559]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:00.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.267+0000 7ff55f7c3700 1 -- 172.21.15.3:0/1459041528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 msgr2=0x7ff558106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.267+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/1459041528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff558106fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff5480099c0 tx=0x7ff548009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 -- 172.21.15.3:0/1459041528 shutdown_connections 2024-05-08T05:49:00.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/1459041528 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5581076c0 0x7ff55810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/1459041528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff558102630 0x7ff558102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/1459041528 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff558106fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 -- 172.21.15.3:0/1459041528 >> 172.21.15.3:0/1459041528 conn(0x7ff558079350 msgr2=0x7ff558079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 -- 172.21.15.3:0/1459041528 shutdown_connections 2024-05-08T05:49:00.269 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 -- 172.21.15.3:0/1459041528 wait complete. 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.268+0000 7ff55f7c3700 1 Processor -- start 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 -- start start 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff558102630 0x7ff55819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff55819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5581076c0 0x7ff5581a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff558112550 con 0x7ff558102fe0 2024-05-08T05:49:00.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff55819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff55819af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59164/0 (socket says 172.21.15.3:59164) 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 -- 172.21.15.3:0/3325840253 learned_addr learned my addr 172.21.15.3:0/3325840253 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581123d0 con 0x7ff5581076c0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5581126d0 con 0x7ff558102630 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55d55f700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff558102630 0x7ff55819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55dd60700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5581076c0 0x7ff5581a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 -- 172.21.15.3:0/3325840253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff558102630 msgr2=0x7ff55819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff558102630 0x7ff55819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 -- 172.21.15.3:0/3325840253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5581076c0 msgr2=0x7ff5581a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5581076c0 0x7ff5581a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.269+0000 7ff55cd5e700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5581a5fd0 con 0x7ff558102fe0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.270+0000 7ff55cd5e700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff55819af50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff548006010 tx=0x7ff5480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.270+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548027070 con 0x7ff558102fe0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.270+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff548003f20 con 0x7ff558102fe0 2024-05-08T05:49:00.273 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.270+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5581a6200 con 0x7ff558102fe0 2024-05-08T05:49:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.270+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548022c60 con 0x7ff558102fe0 2024-05-08T05:49:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.270+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5581a66b0 con 0x7ff558102fe0 2024-05-08T05:49:00.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.271+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55806a190 con 0x7ff558102fe0 2024-05-08T05:49:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.274+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7ff54802a070 con 0x7ff558102fe0 2024-05-08T05:49:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.274+0000 7ff54e7fc700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff5440b2900 0x7ff5440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.274+0000 7ff55d55f700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff5440b2900 0x7ff5440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.274+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7ff548022e00 con 0x7ff558102fe0 2024-05-08T05:49:00.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.275+0000 7ff55d55f700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff5440b2900 0x7ff5440b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff554004630 tx=0x7ff55400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:00.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.278+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff548013070 con 0x7ff558102fe0 2024-05-08T05:49:00.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.490+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff55802e510 con 0x7ff558102fe0 2024-05-08T05:49:00.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.490+0000 7ff54e7fc700 1 -- 172.21.15.3:0/3325840253 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7ff5480ab090 con 0x7ff558102fe0 2024-05-08T05:49:00.490 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff5440b2900 msgr2=0x7ff5440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff5440b2900 0x7ff5440b4dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff554004630 tx=0x7ff55400b3c0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 msgr2=0x7ff55819af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff55819af50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff548006010 tx=0x7ff5480047b0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 shutdown_connections 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff5581076c0 0x7ff5581a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff558102630 0x7ff55819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff5440b2900 0x7ff5440b4dc0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.492+0000 7ff55f7c3700 1 --2- 172.21.15.3:0/3325840253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff558102fe0 0x7ff55819af50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.493+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 >> 172.21.15.3:0/3325840253 conn(0x7ff558079350 msgr2=0x7ff558100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.493+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 shutdown_connections 2024-05-08T05:49:00.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:00.493+0000 7ff55f7c3700 1 -- 172.21.15.3:0/3325840253 wait complete. 2024-05-08T05:49:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:00 smithi003 ceph-mon[30137]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:00 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3325840253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:49:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:00 smithi089 ceph-mon[34495]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:00 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3325840253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:49:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:00 smithi104 ceph-mon[34559]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:00 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3325840253' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.a on smithi003 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.d on smithi003 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.g on smithi003 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.j on smithi003 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.b on smithi089 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.e on smithi089 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.h on smithi089 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.k on smithi089 2024-05-08T05:49:01.175 INFO:tasks.cephadm:Adding mds.c on smithi104 2024-05-08T05:49:01.176 INFO:tasks.cephadm:Adding mds.f on smithi104 2024-05-08T05:49:01.176 INFO:tasks.cephadm:Adding mds.i on smithi104 2024-05-08T05:49:01.176 INFO:tasks.cephadm:Adding mds.l on smithi104 2024-05-08T05:49:01.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l' 2024-05-08T05:49:01.427 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:49:02.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:02 smithi104 ceph-mon[34559]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:02 smithi003 ceph-mon[30137]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:02 smithi089 ceph-mon[34495]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 -- 172.21.15.104:0/388108425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8103510 msgr2=0x7f7ad8103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 --2- 172.21.15.104:0/388108425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8103510 0x7f7ad8103930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009990 tx=0x7f7ac0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 -- 172.21.15.104:0/388108425 shutdown_connections 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 --2- 172.21.15.104:0/388108425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad81050b0 0x7f7ad810b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 --2- 172.21.15.104:0/388108425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8103510 0x7f7ad8103930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 --2- 172.21.15.104:0/388108425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad8104710 0x7f7ad8104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 -- 172.21.15.104:0/388108425 >> 172.21.15.104:0/388108425 conn(0x7f7ad80feab0 msgr2=0x7f7ad8100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 -- 172.21.15.104:0/388108425 shutdown_connections 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.055+0000 7f7add4f2700 1 -- 172.21.15.104:0/388108425 wait complete. 2024-05-08T05:49:03.059 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 Processor -- start 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 -- start start 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad8103510 0x7f7ad8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 0x7f7ad8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad81050b0 0x7f7ad81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad810f010 con 0x7f7ad81050b0 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad810ee90 con 0x7f7ad8103510 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7add4f2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad810f190 con 0x7f7ad8104710 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7ad67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 0x7f7ad8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7ad67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 0x7f7ad8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:55178/0 (socket says 172.21.15.104:55178) 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.056+0000 7f7ad67fc700 1 -- 172.21.15.104:0/4262969707 learned_addr learned my addr 172.21.15.104:0/4262969707 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad6ffd700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad8103510 0x7f7ad8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad77fe700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad81050b0 0x7f7ad81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad67fc700 1 -- 172.21.15.104:0/4262969707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad8103510 msgr2=0x7f7ad8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad67fc700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad8103510 0x7f7ad8199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.060 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad67fc700 1 -- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad81050b0 msgr2=0x7f7ad81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad67fc700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad81050b0 0x7f7ad81a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad67fc700 1 -- 172.21.15.104:0/4262969707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad81a1260 con 0x7f7ad8104710 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7ad67fc700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 0x7f7ad8199b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800c9e0 tx=0x7f7ac800cda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8007ac0 con 0x7f7ad8104710 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac8007c60 con 0x7f7ad8104710 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad81a1550 con 0x7f7ad8104710 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8018660 con 0x7f7ad8104710 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.057+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad81a1a90 con 0x7f7ad8104710 2024-05-08T05:49:03.061 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.058+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad806a190 con 0x7f7ad8104710 2024-05-08T05:49:03.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.060+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7ac800f450 con 0x7f7ad8104710 2024-05-08T05:49:03.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.061+0000 7f7acffff700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7ac40b2880 0x7f7ac40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:03.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.061+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f7ac80e21a0 con 0x7f7ad8104710 2024-05-08T05:49:03.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.061+0000 7f7ad6ffd700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7ac40b2880 0x7f7ac40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:03.062 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.062+0000 7f7ad6ffd700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7ac40b2880 0x7f7ac40b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0006010 tx=0x7f7ac0003860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:03.064 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.064+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7ac8014030 con 0x7f7ad8104710 2024-05-08T05:49:03.277 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.277+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}) -- 0x7f7ad8063ae0 con 0x7f7ac40b2880 2024-05-08T05:49:03.285 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.285+0000 7f7acffff700 1 -- 172.21.15.104:0/4262969707 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7ad8063ae0 con 0x7f7ac40b2880 2024-05-08T05:49:03.285 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mds.all update... 2024-05-08T05:49:03.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7ac40b2880 msgr2=0x7f7ac40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.287 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7ac40b2880 0x7f7ac40b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0006010 tx=0x7f7ac0003860 comp rx=0 tx=0).stop 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 msgr2=0x7f7ad8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 0x7f7ad8199b40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800c9e0 tx=0x7f7ac800cda0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 shutdown_connections 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7ac40b2880 0x7f7ac40b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad81050b0 0x7f7ad81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad8103510 0x7f7ad8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 --2- 172.21.15.104:0/4262969707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad8104710 0x7f7ad8199b40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 >> 172.21.15.104:0/4262969707 conn(0x7f7ad80feab0 msgr2=0x7f7ad8107a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 shutdown_connections 2024-05-08T05:49:03.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-05-08T05:49:03.287+0000 7f7add4f2700 1 -- 172.21.15.104:0/4262969707 wait complete. 2024-05-08T05:49:03.938 DEBUG:teuthology.orchestra.run.smithi003:mds.a> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T05:49:03.942 DEBUG:teuthology.orchestra.run.smithi003:mds.d> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T05:49:03.985 DEBUG:teuthology.orchestra.run.smithi003:mds.g> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T05:49:04.028 DEBUG:teuthology.orchestra.run.smithi003:mds.j> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T05:49:04.071 DEBUG:teuthology.orchestra.run.smithi089:mds.b> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T05:49:04.074 DEBUG:teuthology.orchestra.run.smithi089:mds.e> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T05:49:04.117 DEBUG:teuthology.orchestra.run.smithi089:mds.h> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T05:49:04.160 DEBUG:teuthology.orchestra.run.smithi089:mds.k> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T05:49:04.204 DEBUG:teuthology.orchestra.run.smithi104:mds.c> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T05:49:04.206 DEBUG:teuthology.orchestra.run.smithi104:mds.f> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T05:49:04.250 DEBUG:teuthology.orchestra.run.smithi104:mds.i> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T05:49:04.293 DEBUG:teuthology.orchestra.run.smithi104:mds.l> sudo journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T05:49:04.336 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T05:49:04.336 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:04.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:04.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:04 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:04.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:04.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:04 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:04.589 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: Saving service mds.all spec with placement smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:04.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:04.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:04.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:04 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:05.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:05 smithi003 ceph-mon[30137]: Deploying daemon mds.l on smithi104 2024-05-08T05:49:05.758 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:05 smithi104 ceph-mon[34559]: Deploying daemon mds.l on smithi104 2024-05-08T05:49:05.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:05 smithi089 ceph-mon[34495]: Deploying daemon mds.l on smithi104 2024-05-08T05:49:06.443 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:06 smithi104 ceph-mon[34559]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:06.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 -- 172.21.15.3:0/2921557178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 msgr2=0x7efe801087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 --2- 172.21.15.3:0/2921557178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801087f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b0a0 tx=0x7efe7c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 -- 172.21.15.3:0/2921557178 shutdown_connections 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 --2- 172.21.15.3:0/2921557178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe80102d60 0x7efe8010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 --2- 172.21.15.3:0/2921557178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801087f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 --2- 172.21.15.3:0/2921557178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 0x7efe80102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 -- 172.21.15.3:0/2921557178 >> 172.21.15.3:0/2921557178 conn(0x7efe80079350 msgr2=0x7efe80079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 -- 172.21.15.3:0/2921557178 shutdown_connections 2024-05-08T05:49:06.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.443+0000 7efe87c65700 1 -- 172.21.15.3:0/2921557178 wait complete. 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 Processor -- start 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 -- start start 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 0x7efe8019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe80102d60 0x7efe8019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efe80112560 con 0x7efe80108410 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe801123e0 con 0x7efe801023a0 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe87c65700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efe801126e0 con 0x7efe80102d60 2024-05-08T05:49:06.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe86202700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe86202700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49448/0 (socket says 172.21.15.3:49448) 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe86202700 1 -- 172.21.15.3:0/3231243031 learned_addr learned my addr 172.21.15.3:0/3231243031 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe85200700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe80102d60 0x7efe8019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe85a01700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 0x7efe8019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe86202700 1 -- 172.21.15.3:0/3231243031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe80102d60 msgr2=0x7efe8019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe86202700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe80102d60 0x7efe8019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.444+0000 7efe86202700 1 -- 172.21.15.3:0/3231243031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 msgr2=0x7efe8019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe86202700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 0x7efe8019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe86202700 1 -- 172.21.15.3:0/3231243031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8019a130 con 0x7efe80108410 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe85a01700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 0x7efe8019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe86202700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801999d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efe7800d980 tx=0x7efe7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78017070 con 0x7efe80108410 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe78005020 con 0x7efe80108410 2024-05-08T05:49:06.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780099b0 con 0x7efe80108410 2024-05-08T05:49:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8019a420 con 0x7efe80108410 2024-05-08T05:49:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.445+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe801a66e0 con 0x7efe80108410 2024-05-08T05:49:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.446+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8006a190 con 0x7efe80108410 2024-05-08T05:49:06.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.447+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7efe78004a30 con 0x7efe80108410 2024-05-08T05:49:06.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.448+0000 7efe76ffd700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efe6c0b2880 0x7efe6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:06.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.448+0000 7efe85a01700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efe6c0b2880 0x7efe6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:06.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.448+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7efe780e2020 con 0x7efe80108410 2024-05-08T05:49:06.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.449+0000 7efe85a01700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efe6c0b2880 0x7efe6c0b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0062a0 tx=0x7efe7c006230 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:06.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.452+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efe78020020 con 0x7efe80108410 2024-05-08T05:49:06.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:06 smithi003 ceph-mon[30137]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.713+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe801a6910 con 0x7efe80108410 2024-05-08T05:49:06.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.713+0000 7efe76ffd700 1 -- 172.21.15.3:0/3231243031 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7efe780aa140 con 0x7efe80108410 2024-05-08T05:49:06.713 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:06.713 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efe6c0b2880 msgr2=0x7efe6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efe6c0b2880 0x7efe6c0b4d40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0062a0 tx=0x7efe7c006230 comp rx=0 tx=0).stop 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 msgr2=0x7efe801999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801999d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efe7800d980 tx=0x7efe7800dc90 comp rx=0 tx=0).stop 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 shutdown_connections 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efe6c0b2880 0x7efe6c0b4d40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efe80108410 0x7efe801999d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe801023a0 0x7efe8019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 --2- 172.21.15.3:0/3231243031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe80102d60 0x7efe8019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 >> 172.21.15.3:0/3231243031 conn(0x7efe80079350 msgr2=0x7efe801017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 shutdown_connections 2024-05-08T05:49:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:06.716+0000 7efe87c65700 1 -- 172.21.15.3:0/3231243031 wait complete. 2024-05-08T05:49:06.717 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 1 2024-05-08T05:49:06.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:06 smithi089 ceph-mon[34495]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:07.021 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 05:49:06 smithi104 systemd[1]: Started Ceph mds.l for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:07.655 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3231243031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.104:6832/1254707053,v1:172.21.15.104:6833/1254707053] up:boot 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: fsmap 1 up:standby 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:07.656 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:07 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3231243031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.104:6832/1254707053,v1:172.21.15.104:6833/1254707053] up:boot 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: fsmap 1 up:standby 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:07.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:07 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3231243031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.104:6832/1254707053,v1:172.21.15.104:6833/1254707053] up:boot 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: fsmap 1 up:standby 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:07.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:07.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:07.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:07 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:08.545 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:08 smithi089 ceph-mon[34495]: Deploying daemon mds.k on smithi089 2024-05-08T05:49:08.545 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:08 smithi089 ceph-mon[34495]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:08 smithi003 ceph-mon[30137]: Deploying daemon mds.k on smithi089 2024-05-08T05:49:08.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:08 smithi003 ceph-mon[30137]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:08.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:08 smithi104 ceph-mon[34559]: Deploying daemon mds.k on smithi089 2024-05-08T05:49:08.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:08 smithi104 ceph-mon[34559]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:09.262 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 systemd[1]: Starting Ceph mds.k for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:09.482 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:09.599 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 podman[64183]: 2024-05-08 05:49:09.247348999 +0000 UTC m=+0.017921586 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:09.599 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 podman[64183]: 2024-05-08 05:49:09.345783125 +0000 UTC m=+0.116355705 container create 37dbc486ad7de9b9494a535d89ad8f4cde7ab94433e9fe46ac690f28180cd118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-08T05:49:09.599 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 podman[64183]: 2024-05-08 05:49:09.573805582 +0000 UTC m=+0.344378166 container init 37dbc486ad7de9b9494a535d89ad8f4cde7ab94433e9fe46ac690f28180cd118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-08T05:49:09.599 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 podman[64183]: 2024-05-08 05:49:09.576450722 +0000 UTC m=+0.347023303 container start 37dbc486ad7de9b9494a535d89ad8f4cde7ab94433e9fe46ac690f28180cd118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-08T05:49:09.744 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:10.013 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 ceph-mds[64203]: -- 172.21.15.89:0/2376190820 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfece9da40 con 0x55dfece70800 2024-05-08T05:49:10.013 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k[64199]: starting mds.k at 2024-05-08T05:49:10.013 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 bash[64183]: 37dbc486ad7de9b9494a535d89ad8f4cde7ab94433e9fe46ac690f28180cd118 2024-05-08T05:49:10.013 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 05:49:09 smithi089 systemd[1]: Started Ceph mds.k for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:10.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:10 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:10.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:10 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:10.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:10 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:11.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:11 smithi003 ceph-mon[30137]: Deploying daemon mds.j on smithi003 2024-05-08T05:49:11.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:11 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.89:6832/3884583981,v1:172.21.15.89:6833/3884583981] up:boot 2024-05-08T05:49:11.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:11 smithi003 ceph-mon[30137]: fsmap 2 up:standby 2024-05-08T05:49:11.444 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:11 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:49:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.548+0000 7f46753a7700 1 -- 172.21.15.3:0/2713052005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 msgr2=0x7f467010f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.548+0000 7f46753a7700 1 --2- 172.21.15.3:0/2713052005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467010f9b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4664009990 tx=0x7f4664009ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 -- 172.21.15.3:0/2713052005 shutdown_connections 2024-05-08T05:49:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 --2- 172.21.15.3:0/2713052005 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467010fef0 0x7f46701122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 --2- 172.21.15.3:0/2713052005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467006bc50 0x7f467006c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 --2- 172.21.15.3:0/2713052005 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467010f9b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 -- 172.21.15.3:0/2713052005 >> 172.21.15.3:0/2713052005 conn(0x7f4670079440 msgr2=0x7f4670079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 -- 172.21.15.3:0/2713052005 shutdown_connections 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 -- 172.21.15.3:0/2713052005 wait complete. 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 Processor -- start 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.549+0000 7f46753a7700 1 -- start start 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f46753a7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467006bc50 0x7f467019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f46753a7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467019c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f46753a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467010fef0 0x7f46701a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f46753a7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4670115920 con 0x7f467006c570 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f46753a7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f46701157a0 con 0x7f467006bc50 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f46753a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4670115aa0 con 0x7f467010fef0 2024-05-08T05:49:11.553 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467019c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467019c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49476/0 (socket says 172.21.15.3:49476) 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467006bc50 0x7f467019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 -- 172.21.15.3:0/3895218966 learned_addr learned my addr 172.21.15.3:0/3895218966 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466f7fe700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467010fef0 0x7f46701a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 -- 172.21.15.3:0/3895218966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467010fef0 msgr2=0x7f46701a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467010fef0 0x7f46701a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 -- 172.21.15.3:0/3895218966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467006bc50 msgr2=0x7f467019be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467006bc50 0x7f467019be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 -- 172.21.15.3:0/3895218966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46701a0e50 con 0x7f467006c570 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.550+0000 7f466e7fc700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467019c3d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4664000c00 tx=0x7f46640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.551+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664027070 con 0x7f467006c570 2024-05-08T05:49:11.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.551+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4664003f20 con 0x7f467006c570 2024-05-08T05:49:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.551+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46701a10e0 con 0x7f467006c570 2024-05-08T05:49:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.551+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664022cf0 con 0x7f467006c570 2024-05-08T05:49:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.551+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46701a89c0 con 0x7f467006c570 2024-05-08T05:49:11.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.551+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467006a190 con 0x7f467006c570 2024-05-08T05:49:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.553+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f466402a070 con 0x7f467006c570 2024-05-08T05:49:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.554+0000 7f4657fff700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46580b2880 0x7f46580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.554+0000 7f466effd700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46580b2880 0x7f46580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.554+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f46640e2eb0 con 0x7f467006c570 2024-05-08T05:49:11.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.555+0000 7f466effd700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46580b2880 0x7f46580b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4660009a10 tx=0x7f4660009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:11.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.558+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4664013070 con 0x7f467006c570 2024-05-08T05:49:11.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:11 smithi089 ceph-mon[34495]: Deploying daemon mds.j on smithi003 2024-05-08T05:49:11.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:11 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.89:6832/3884583981,v1:172.21.15.89:6833/3884583981] up:boot 2024-05-08T05:49:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:11 smithi089 ceph-mon[34495]: fsmap 2 up:standby 2024-05-08T05:49:11.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:11 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:49:11.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:11 smithi104 ceph-mon[34559]: Deploying daemon mds.j on smithi003 2024-05-08T05:49:11.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:11 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.89:6832/3884583981,v1:172.21.15.89:6833/3884583981] up:boot 2024-05-08T05:49:11.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:11 smithi104 ceph-mon[34559]: fsmap 2 up:standby 2024-05-08T05:49:11.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:11 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:49:11.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.822+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f467002e510 con 0x7f467006c570 2024-05-08T05:49:11.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.823+0000 7f4657fff700 1 -- 172.21.15.3:0/3895218966 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f46640ab100 con 0x7f467006c570 2024-05-08T05:49:11.823 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:11.823 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46580b2880 msgr2=0x7f46580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46580b2880 0x7f46580b4d40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4660009a10 tx=0x7f4660009190 comp rx=0 tx=0).stop 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 msgr2=0x7f467019c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467019c3d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4664000c00 tx=0x7f46640047b0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 shutdown_connections 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f467010fef0 0x7f46701a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f467006bc50 0x7f467019be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46580b2880 0x7f46580b4d40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 --2- 172.21.15.3:0/3895218966 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f467006c570 0x7f467019c3d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:11.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.825+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 >> 172.21.15.3:0/3895218966 conn(0x7f4670079440 msgr2=0x7f46701114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.826+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 shutdown_connections 2024-05-08T05:49:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:11.826+0000 7f46753a7700 1 -- 172.21.15.3:0/3895218966 wait complete. 2024-05-08T05:49:11.827 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 3 2024-05-08T05:49:12.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:12 smithi003 ceph-mon[30137]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:12.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:12 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3895218966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:12.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:12 smithi089 ceph-mon[34495]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:12.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:12 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3895218966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:12.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:12 smithi104 ceph-mon[34559]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:12.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:12 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3895218966' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:14.160 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:13 smithi003 systemd[1]: Starting Ceph mds.j for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:14.509 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:14 smithi003 ceph-mon[30137]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:14.510 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 podman[70872]: 2024-05-08 05:49:14.121842856 +0000 UTC m=+0.018160519 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:14.510 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 podman[70872]: 2024-05-08 05:49:14.233258409 +0000 UTC m=+0.129576054 container create 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, ceph=True) 2024-05-08T05:49:14.510 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 podman[70872]: 2024-05-08 05:49:14.510571999 +0000 UTC m=+0.406889650 container init 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-08T05:49:14.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:14 smithi089 ceph-mon[34495]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:14.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:14 smithi104 ceph-mon[34559]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:14.910 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 podman[70872]: 2024-05-08 05:49:14.513676226 +0000 UTC m=+0.409993875 container start 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:49:14.910 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 ceph-mds[70891]: -- 172.21.15.3:0/1708825450 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585e1c87a40 con 0x5585e1c5b000 2024-05-08T05:49:14.910 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j[70887]: starting mds.j at 2024-05-08T05:49:14.910 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 bash[70872]: 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 2024-05-08T05:49:14.910 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 05:49:14 smithi003 systemd[1]: Started Ceph mds.j for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:15.023 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:15.529 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: Deploying daemon mds.d on smithi003 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.3:6834/2960751422,v1:172.21.15.3:6835/2960751422] up:boot 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: fsmap 3 up:standby 2024-05-08T05:49:16.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:15 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: Deploying daemon mds.d on smithi003 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.3:6834/2960751422,v1:172.21.15.3:6835/2960751422] up:boot 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: fsmap 3 up:standby 2024-05-08T05:49:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:15 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:49:16.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.132 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: Deploying daemon mds.d on smithi003 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.3:6834/2960751422,v1:172.21.15.3:6835/2960751422] up:boot 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: fsmap 3 up:standby 2024-05-08T05:49:16.133 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:15 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:49:16.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:16 smithi003 ceph-mon[30137]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:16 smithi089 ceph-mon[34495]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:16 smithi104 ceph-mon[34559]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:17.247 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:17 smithi003 systemd[1]: Starting Ceph mds.d for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 -- 172.21.15.3:0/575332083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 msgr2=0x7f2d5c1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 --2- 172.21.15.3:0/575332083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c1103f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400b0d0 tx=0x7f2d5400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 -- 172.21.15.3:0/575332083 shutdown_connections 2024-05-08T05:49:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 --2- 172.21.15.3:0/575332083 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c1103f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 --2- 172.21.15.3:0/575332083 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c103780 0x7f2d5c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 --2- 172.21.15.3:0/575332083 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d5c104130 0x7f2d5c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 -- 172.21.15.3:0/575332083 >> 172.21.15.3:0/575332083 conn(0x7f2d5c07a610 msgr2=0x7f2d5c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 -- 172.21.15.3:0/575332083 shutdown_connections 2024-05-08T05:49:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.160+0000 7f2d62fef700 1 -- 172.21.15.3:0/575332083 wait complete. 2024-05-08T05:49:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 Processor -- start 2024-05-08T05:49:18.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 -- start start 2024-05-08T05:49:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d5c103780 0x7f2d5c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c104130 0x7f2d5c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c113660 con 0x7f2d5c108870 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1134e0 con 0x7f2d5c104130 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d62fef700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5c1137e0 con 0x7f2d5c103780 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d6158c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.161+0000 7f2d6158c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54108/0 (socket says 172.21.15.3:54108) 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 -- 172.21.15.3:0/3554337436 learned_addr learned my addr 172.21.15.3:0/3554337436 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d5bfff700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c104130 0x7f2d5c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d60d8b700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d5c103780 0x7f2d5c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 -- 172.21.15.3:0/3554337436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d5c103780 msgr2=0x7f2d5c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d5c103780 0x7f2d5c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 -- 172.21.15.3:0/3554337436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c104130 msgr2=0x7f2d5c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c104130 0x7f2d5c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 -- 172.21.15.3:0/3554337436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c19a080 con 0x7f2d5c108870 2024-05-08T05:49:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d6158c700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c199920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d540095c0 tx=0x7f2d54007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5401f070 con 0x7f2d5c108870 2024-05-08T05:49:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d54007e20 con 0x7f2d5c108870 2024-05-08T05:49:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5401b4a0 con 0x7f2d5c108870 2024-05-08T05:49:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.162+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5c19a310 con 0x7f2d5c108870 2024-05-08T05:49:18.165 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.163+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d5c1a6630 con 0x7f2d5c108870 2024-05-08T05:49:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.164+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d5c06a190 con 0x7f2d5c108870 2024-05-08T05:49:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.165+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2d54022070 con 0x7f2d5c108870 2024-05-08T05:49:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.166+0000 7f2d59ffb700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2d4c0b2900 0x7f2d4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.166+0000 7f2d60d8b700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2d4c0b2900 0x7f2d4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.166+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f2d5400f020 con 0x7f2d5c108870 2024-05-08T05:49:18.166 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.166+0000 7f2d60d8b700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2d4c0b2900 0x7f2d4c0b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c19aa60 tx=0x7f2d5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:18.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.170+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d54029020 con 0x7f2d5c108870 2024-05-08T05:49:18.454 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.454+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d5c02e510 con 0x7f2d5c108870 2024-05-08T05:49:18.454 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:18 smithi003 podman[71317]: 2024-05-08 05:49:18.242823067 +0000 UTC m=+0.021308762 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:18.455 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:18 smithi003 podman[71317]: 2024-05-08 05:49:18.379734449 +0000 UTC m=+0.158220109 container create b97c61b19939e616bbd10196de10d5561b5fca92f9e9b7847826d9b456e299ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T05:49:18.455 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.454+0000 7f2d59ffb700 1 -- 172.21.15.3:0/3554337436 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f2d540ab4e0 con 0x7f2d5c108870 2024-05-08T05:49:18.455 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:18.455 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2d4c0b2900 msgr2=0x7f2d4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2d4c0b2900 0x7f2d4c0b4dc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c19aa60 tx=0x7f2d5000b410 comp rx=0 tx=0).stop 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 msgr2=0x7f2d5c199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c199920 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2d540095c0 tx=0x7f2d54007a20 comp rx=0 tx=0).stop 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 shutdown_connections 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2d4c0b2900 0x7f2d4c0b4dc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2d5c108870 0x7f2d5c199920 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d5c103780 0x7f2d5c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 --2- 172.21.15.3:0/3554337436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2d5c104130 0x7f2d5c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:18.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.456+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 >> 172.21.15.3:0/3554337436 conn(0x7f2d5c07a610 msgr2=0x7f2d5c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:18.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.457+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 shutdown_connections 2024-05-08T05:49:18.458 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:18.457+0000 7f2d62fef700 1 -- 172.21.15.3:0/3554337436 wait complete. 2024-05-08T05:49:18.458 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 4 2024-05-08T05:49:19.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:18 smithi089 ceph-mon[34495]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:18 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3554337436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:18 smithi104 ceph-mon[34559]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:18 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3554337436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:19.085 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:18 smithi003 ceph-mon[30137]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:19.085 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:18 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3554337436' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:19.386 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:19 smithi003 podman[71317]: 2024-05-08 05:49:19.286366909 +0000 UTC m=+1.064852580 container init b97c61b19939e616bbd10196de10d5561b5fca92f9e9b7847826d9b456e299ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-08T05:49:19.386 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:19 smithi003 podman[71317]: 2024-05-08 05:49:19.289691294 +0000 UTC m=+1.068176969 container start b97c61b19939e616bbd10196de10d5561b5fca92f9e9b7847826d9b456e299ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:49:19.386 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:19 smithi003 ceph-mds[71353]: -- 172.21.15.3:0/180044709 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556910431a40 con 0x556910404c00 2024-05-08T05:49:19.386 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d[71348]: starting mds.d at 2024-05-08T05:49:19.660 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:19 smithi003 bash[71317]: b97c61b19939e616bbd10196de10d5561b5fca92f9e9b7847826d9b456e299ca 2024-05-08T05:49:19.660 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 05:49:19 smithi003 systemd[1]: Started Ceph mds.d for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: Deploying daemon mds.e on smithi089 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.3:6836/2285392354,v1:172.21.15.3:6837/2285392354] up:boot 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: fsmap 4 up:standby 2024-05-08T05:49:20.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:20 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:20.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:20.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: Deploying daemon mds.e on smithi089 2024-05-08T05:49:20.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.3:6836/2285392354,v1:172.21.15.3:6837/2285392354] up:boot 2024-05-08T05:49:20.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: fsmap 4 up:standby 2024-05-08T05:49:20.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:20 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:20.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: Deploying daemon mds.e on smithi089 2024-05-08T05:49:20.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.3:6836/2285392354,v1:172.21.15.3:6837/2285392354] up:boot 2024-05-08T05:49:20.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: fsmap 4 up:standby 2024-05-08T05:49:20.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:20 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:49:21.114 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:21.378 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:21.903 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:21 smithi089 systemd[1]: Starting Ceph mds.e for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:22.196 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:21 smithi089 podman[64501]: 2024-05-08 05:49:21.902305945 +0000 UTC m=+0.108220975 container create 770bed1fe3197a6742abaee6f84baf6a035e2bc4104e31f49a9cb905cf407e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T05:49:22.197 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:21 smithi089 podman[64501]: 2024-05-08 05:49:21.812226903 +0000 UTC m=+0.018141895 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:22.197 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:22 smithi089 podman[64501]: 2024-05-08 05:49:22.121586687 +0000 UTC m=+0.327501674 container init 770bed1fe3197a6742abaee6f84baf6a035e2bc4104e31f49a9cb905cf407e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507) 2024-05-08T05:49:22.197 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:22 smithi089 podman[64501]: 2024-05-08 05:49:22.12402093 +0000 UTC m=+0.329935924 container start 770bed1fe3197a6742abaee6f84baf6a035e2bc4104e31f49a9cb905cf407e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:49:22.197 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mds[64520]: -- 172.21.15.89:0/2870134959 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ed011fa40 con 0x557ed00f2c00 2024-05-08T05:49:22.197 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e[64516]: starting mds.e at 2024-05-08T05:49:22.197 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:22 smithi089 bash[64501]: 770bed1fe3197a6742abaee6f84baf6a035e2bc4104e31f49a9cb905cf407e2c 2024-05-08T05:49:22.446 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 05:49:22 smithi089 systemd[1]: Started Ceph mds.e for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:22.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:22.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.716 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:22.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:22.717 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:22 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:22.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:22.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:22.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:22.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:22 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:22.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:22 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:23.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 -- 172.21.15.3:0/1001128392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae641037b0 msgr2=0x7fae64103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 --2- 172.21.15.3:0/1001128392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae641037b0 0x7fae64103b90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fae540099c0 tx=0x7fae54009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 -- 172.21.15.3:0/1001128392 shutdown_connections 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 --2- 172.21.15.3:0/1001128392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae64108880 0x7fae64110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 --2- 172.21.15.3:0/1001128392 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae641037b0 0x7fae64103b90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 --2- 172.21.15.3:0/1001128392 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae64104160 0x7fae64108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 -- 172.21.15.3:0/1001128392 >> 172.21.15.3:0/1001128392 conn(0x7fae6407a580 msgr2=0x7fae6407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 -- 172.21.15.3:0/1001128392 shutdown_connections 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.263+0000 7fae69db7700 1 -- 172.21.15.3:0/1001128392 wait complete. 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 Processor -- start 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 -- start start 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae641037b0 0x7fae640757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 0x7fae64075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae64108880 0x7fae64076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae64113670 con 0x7fae64104160 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fae641134f0 con 0x7fae641037b0 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae69db7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae641137f0 con 0x7fae64108880 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 0x7fae64075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae62ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 0x7fae64075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54762/0 (socket says 172.21.15.3:54762) 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.264+0000 7fae62ffd700 1 -- 172.21.15.3:0/3521233885 learned_addr learned my addr 172.21.15.3:0/3521233885 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae63fff700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae64108880 0x7fae64076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae637fe700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae641037b0 0x7fae640757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae62ffd700 1 -- 172.21.15.3:0/3521233885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae64108880 msgr2=0x7fae64076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae62ffd700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae64108880 0x7fae64076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae62ffd700 1 -- 172.21.15.3:0/3521233885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae641037b0 msgr2=0x7fae640757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae62ffd700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae641037b0 0x7fae640757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae62ffd700 1 -- 172.21.15.3:0/3521233885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae641a8030 con 0x7fae64104160 2024-05-08T05:49:23.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae62ffd700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 0x7fae64075ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d980 tx=0x7fae5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58016070 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae58004830 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae641a8320 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae58005540 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.265+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae641a88b0 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.266+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae6410bac0 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.269+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fae58004220 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.269+0000 7fae60ff9700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae4c0b2900 0x7fae4c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.269+0000 7fae637fe700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae4c0b2900 0x7fae4c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.270+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fae580e1fc0 con 0x7fae64104160 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.270+0000 7fae637fe700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae4c0b2900 0x7fae4c0b4dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae54009990 tx=0x7fae54009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:23.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.273+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae5801f020 con 0x7fae64104160 2024-05-08T05:49:23.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.537+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae6410b4a0 con 0x7fae64104160 2024-05-08T05:49:23.538 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:23 smithi003 ceph-mon[30137]: Deploying daemon mds.f on smithi104 2024-05-08T05:49:23.538 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:23 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.89:6834/3820045271,v1:172.21.15.89:6835/3820045271] up:boot 2024-05-08T05:49:23.538 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:23 smithi003 ceph-mon[30137]: fsmap 5 up:standby 2024-05-08T05:49:23.538 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:23 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:49:23.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.538+0000 7fae60ff9700 1 -- 172.21.15.3:0/3521233885 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7fae580b2030 con 0x7fae64104160 2024-05-08T05:49:23.539 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:23.539 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.540+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae4c0b2900 msgr2=0x7fae4c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.540+0000 7fae69db7700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae4c0b2900 0x7fae4c0b4dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fae54009990 tx=0x7fae54009650 comp rx=0 tx=0).stop 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.540+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 msgr2=0x7fae64075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 0x7fae64075ce0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d980 tx=0x7fae5800dc90 comp rx=0 tx=0).stop 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 shutdown_connections 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae64108880 0x7fae64076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae641037b0 0x7fae640757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae4c0b2900 0x7fae4c0b4dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 --2- 172.21.15.3:0/3521233885 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae64104160 0x7fae64075ce0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:23.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 >> 172.21.15.3:0/3521233885 conn(0x7fae6407a580 msgr2=0x7fae641099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:23.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 shutdown_connections 2024-05-08T05:49:23.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:23.541+0000 7fae69db7700 1 -- 172.21.15.3:0/3521233885 wait complete. 2024-05-08T05:49:23.542 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 6 2024-05-08T05:49:23.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:23 smithi089 ceph-mon[34495]: Deploying daemon mds.f on smithi104 2024-05-08T05:49:23.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:23 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.89:6834/3820045271,v1:172.21.15.89:6835/3820045271] up:boot 2024-05-08T05:49:23.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:23 smithi089 ceph-mon[34495]: fsmap 5 up:standby 2024-05-08T05:49:23.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:23 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:49:23.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:23 smithi104 ceph-mon[34559]: Deploying daemon mds.f on smithi104 2024-05-08T05:49:23.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:23 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.89:6834/3820045271,v1:172.21.15.89:6835/3820045271] up:boot 2024-05-08T05:49:23.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:23 smithi104 ceph-mon[34559]: fsmap 5 up:standby 2024-05-08T05:49:23.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:23 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:49:24.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:24 smithi104 ceph-mon[34559]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:24.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:24 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3521233885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:24.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:24 smithi089 ceph-mon[34495]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:24.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:24 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3521233885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:24 smithi003 ceph-mon[30137]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:24.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:24 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3521233885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:26.230 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: Deploying daemon mds.c on smithi104 2024-05-08T05:49:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:26.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.104:6834/3213984653,v1:172.21.15.104:6835/3213984653] up:boot 2024-05-08T05:49:26.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: fsmap 6 up:standby 2024-05-08T05:49:26.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:26 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: Deploying daemon mds.c on smithi104 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.104:6834/3213984653,v1:172.21.15.104:6835/3213984653] up:boot 2024-05-08T05:49:26.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: fsmap 6 up:standby 2024-05-08T05:49:26.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:26 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:49:26.500 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:26.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: Deploying daemon mds.c on smithi104 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.104:6834/3213984653,v1:172.21.15.104:6835/3213984653] up:boot 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: fsmap 6 up:standby 2024-05-08T05:49:26.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:26 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:49:27.271 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 systemd[1]: Starting Ceph mds.c for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:27.729 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 podman[64971]: 2024-05-08 05:49:27.308250593 +0000 UTC m=+0.017968368 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:27.729 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 podman[64971]: 2024-05-08 05:49:27.443948107 +0000 UTC m=+0.153665881 container create 505603efcfa70fe9e26dcb925d3a4c71558ecec001c5a4e2bf332b9954147095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:49:27.729 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 podman[64971]: 2024-05-08 05:49:27.729846936 +0000 UTC m=+0.439564707 container init 505603efcfa70fe9e26dcb925d3a4c71558ecec001c5a4e2bf332b9954147095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T05:49:28.021 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 podman[64971]: 2024-05-08 05:49:27.732476436 +0000 UTC m=+0.442194209 container start 505603efcfa70fe9e26dcb925d3a4c71558ecec001c5a4e2bf332b9954147095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-08T05:49:28.021 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 ceph-mds[64991]: -- 172.21.15.104:0/1833188957 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56189aae7a40 con 0x56189aaba800 2024-05-08T05:49:28.021 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c[64987]: starting mds.c at 2024-05-08T05:49:28.021 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 bash[64971]: 505603efcfa70fe9e26dcb925d3a4c71558ecec001c5a4e2bf332b9954147095 2024-05-08T05:49:28.021 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 05:49:27 smithi104 systemd[1]: Started Ceph mds.c for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:28.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.230+0000 7f5e72287700 1 -- 172.21.15.3:0/2163596735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c06bad0 msgr2=0x7f5e6c10fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.230+0000 7f5e72287700 1 --2- 172.21.15.3:0/2163596735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c06bad0 0x7f5e6c10fa10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5e54009990 tx=0x7f5e54009ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 -- 172.21.15.3:0/2163596735 shutdown_connections 2024-05-08T05:49:28.233 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 --2- 172.21.15.3:0/2163596735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c10ff50 0x7f5e6c112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 --2- 172.21.15.3:0/2163596735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 --2- 172.21.15.3:0/2163596735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c06bad0 0x7f5e6c10fa10 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 -- 172.21.15.3:0/2163596735 >> 172.21.15.3:0/2163596735 conn(0x7f5e6c079350 msgr2=0x7f5e6c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 -- 172.21.15.3:0/2163596735 shutdown_connections 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 -- 172.21.15.3:0/2163596735 wait complete. 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 Processor -- start 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.231+0000 7f5e72287700 1 -- start start 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e72287700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e72287700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e6c06bad0 0x7f5e6c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e72287700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c10ff50 0x7f5e6c1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e72287700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c115860 con 0x7f5e6c10ff50 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e72287700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1156e0 con 0x7f5e6c06b1b0 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e72287700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1159e0 con 0x7f5e6c06bad0 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e6bfff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:54324/0 (socket says 172.21.15.3:54324) 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c10ff50 0x7f5e6c1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e6bfff700 1 -- 172.21.15.3:0/3684474405 learned_addr learned my addr 172.21.15.3:0/3684474405 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e63fff700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e6c06bad0 0x7f5e6c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 -- 172.21.15.3:0/3684474405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e6c06bad0 msgr2=0x7f5e6c19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e6c06bad0 0x7f5e6c19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 -- 172.21.15.3:0/3684474405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 msgr2=0x7f5e6c19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c19bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 -- 172.21.15.3:0/3684474405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c1a0e00 con 0x7f5e6c10ff50 2024-05-08T05:49:28.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e6bfff700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.232+0000 7f5e70824700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c10ff50 0x7f5e6c1a06c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e64007ee0 tx=0x7f5e6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.233+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e64004db0 con 0x7f5e6c10ff50 2024-05-08T05:49:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.233+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e64003a00 con 0x7f5e6c10ff50 2024-05-08T05:49:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.233+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6c1a10f0 con 0x7f5e6c10ff50 2024-05-08T05:49:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.233+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e640044e0 con 0x7f5e6c10ff50 2024-05-08T05:49:28.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.233+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6c1a8970 con 0x7f5e6c10ff50 2024-05-08T05:49:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.234+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6c1029c0 con 0x7f5e6c10ff50 2024-05-08T05:49:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.235+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5e6401c020 con 0x7f5e6c10ff50 2024-05-08T05:49:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.236+0000 7f5e69ffb700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e580b2880 0x7f5e580b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.236+0000 7f5e6bfff700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e580b2880 0x7f5e580b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.236+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f5e640e1ed0 con 0x7f5e6c10ff50 2024-05-08T05:49:28.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.237+0000 7f5e6bfff700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e580b2880 0x7f5e580b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00f4d0 tx=0x7f5e5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:28.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.240+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e64023030 con 0x7f5e6c10ff50 2024-05-08T05:49:28.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.511+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e6c19d150 con 0x7f5e6c10ff50 2024-05-08T05:49:28.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:28.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.512 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:28.513 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:28.513 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:28 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:28.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.512+0000 7f5e69ffb700 1 -- 172.21.15.3:0/3684474405 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f5e64010070 con 0x7f5e6c10ff50 2024-05-08T05:49:28.513 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:28.513 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:28.514 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:28 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e580b2880 msgr2=0x7f5e580b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e580b2880 0x7f5e580b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00f4d0 tx=0x7f5e5c005f90 comp rx=0 tx=0).stop 2024-05-08T05:49:28.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c10ff50 msgr2=0x7f5e6c1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c10ff50 0x7f5e6c1a06c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5e64007ee0 tx=0x7f5e6400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 shutdown_connections 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e580b2880 0x7f5e580b4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e6c10ff50 0x7f5e6c1a06c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e6c06b1b0 0x7f5e6c19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 --2- 172.21.15.3:0/3684474405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e6c06bad0 0x7f5e6c19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 >> 172.21.15.3:0/3684474405 conn(0x7f5e6c079350 msgr2=0x7f5e6c1114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 shutdown_connections 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:28.515+0000 7f5e72287700 1 -- 172.21.15.3:0/3684474405 wait complete. 2024-05-08T05:49:28.516 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 8 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:28.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:28 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:29 smithi104 ceph-mon[34559]: Deploying daemon mds.h on smithi089 2024-05-08T05:49:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:29 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.104:6836/2023592505,v1:172.21.15.104:6837/2023592505] up:boot 2024-05-08T05:49:29.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:29 smithi104 ceph-mon[34559]: fsmap 7 up:standby 2024-05-08T05:49:29.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:29 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:49:29.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:29 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3684474405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:29.559 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:29 smithi089 ceph-mon[34495]: Deploying daemon mds.h on smithi089 2024-05-08T05:49:29.559 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:29 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.104:6836/2023592505,v1:172.21.15.104:6837/2023592505] up:boot 2024-05-08T05:49:29.559 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:29 smithi089 ceph-mon[34495]: fsmap 7 up:standby 2024-05-08T05:49:29.559 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:29 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:49:29.559 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:29 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3684474405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:29.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:29 smithi003 ceph-mon[30137]: Deploying daemon mds.h on smithi089 2024-05-08T05:49:29.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:29 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.104:6836/2023592505,v1:172.21.15.104:6837/2023592505] up:boot 2024-05-08T05:49:29.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:29 smithi003 ceph-mon[30137]: fsmap 7 up:standby 2024-05-08T05:49:29.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:29 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:49:29.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:29 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3684474405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:30.261 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 systemd[1]: Starting Ceph mds.h for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:30.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:30 smithi089 ceph-mon[34495]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:30.513 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 podman[64811]: 2024-05-08 05:49:30.164588844 +0000 UTC m=+0.017991823 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:30.513 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 podman[64811]: 2024-05-08 05:49:30.287984929 +0000 UTC m=+0.141387890 container create d0ff2251bc9ed2d382e36cc90809c2b03f53883502514e34f2253dfea1bd4346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:49:30.513 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 podman[64811]: 2024-05-08 05:49:30.507350145 +0000 UTC m=+0.360753112 container init d0ff2251bc9ed2d382e36cc90809c2b03f53883502514e34f2253dfea1bd4346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-08T05:49:30.513 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 podman[64811]: 2024-05-08 05:49:30.5098524 +0000 UTC m=+0.363255365 container start d0ff2251bc9ed2d382e36cc90809c2b03f53883502514e34f2253dfea1bd4346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:49:30.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:30 smithi104 ceph-mon[34559]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:30.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:30 smithi003 ceph-mon[30137]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:31.012 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 ceph-mds[64830]: -- 172.21.15.89:0/4120778706 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d2a8b1a40 con 0x555d2a884800 2024-05-08T05:49:31.012 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h[64826]: starting mds.h at 2024-05-08T05:49:31.013 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 bash[64811]: d0ff2251bc9ed2d382e36cc90809c2b03f53883502514e34f2253dfea1bd4346 2024-05-08T05:49:31.013 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 05:49:30 smithi089 systemd[1]: Started Ceph mds.h for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:31.248 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:31.512 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: Deploying daemon mds.i on smithi104 2024-05-08T05:49:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.89:6836/1710942152,v1:172.21.15.89:6837/1710942152] up:boot 2024-05-08T05:49:31.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: fsmap 8 up:standby 2024-05-08T05:49:31.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:31 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:49:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: Deploying daemon mds.i on smithi104 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.89:6836/1710942152,v1:172.21.15.89:6837/1710942152] up:boot 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: fsmap 8 up:standby 2024-05-08T05:49:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:31 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:49:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:32.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: Deploying daemon mds.i on smithi104 2024-05-08T05:49:32.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.89:6836/1710942152,v1:172.21.15.89:6837/1710942152] up:boot 2024-05-08T05:49:32.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: fsmap 8 up:standby 2024-05-08T05:49:32.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:31 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:49:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:32 smithi003 ceph-mon[30137]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:33.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:32 smithi089 ceph-mon[34495]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:32 smithi104 ceph-mon[34559]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:33.021 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:32 smithi104 systemd[1]: Starting Ceph mds.i for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 -- 172.21.15.3:0/1370616006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac108410 msgr2=0x7f81ac1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 --2- 172.21.15.3:0/1370616006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac108410 0x7f81ac1087f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f81a800b0a0 tx=0x7f81a800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 -- 172.21.15.3:0/1370616006 shutdown_connections 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 --2- 172.21.15.3:0/1370616006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac102d60 0x7f81ac10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 --2- 172.21.15.3:0/1370616006 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac108410 0x7f81ac1087f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 --2- 172.21.15.3:0/1370616006 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ac1023a0 0x7f81ac102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.190+0000 7f81b3417700 1 -- 172.21.15.3:0/1370616006 >> 172.21.15.3:0/1370616006 conn(0x7f81ac079350 msgr2=0x7f81ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 -- 172.21.15.3:0/1370616006 shutdown_connections 2024-05-08T05:49:33.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 -- 172.21.15.3:0/1370616006 wait complete. 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 Processor -- start 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 -- start start 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ac1023a0 0x7f81ac19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 0x7f81ac19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.191+0000 7f81b3417700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac108410 0x7f81ac1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b3417700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac112530 con 0x7f81ac102d60 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b3417700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac1123b0 con 0x7f81ac1023a0 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b3417700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81ac1126b0 con 0x7f81ac108410 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 0x7f81ac19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 0x7f81ac19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60590/0 (socket says 172.21.15.3:60590) 2024-05-08T05:49:33.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 -- 172.21.15.3:0/1186449572 learned_addr learned my addr 172.21.15.3:0/1186449572 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b11b3700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ac1023a0 0x7f81ac19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b19b4700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac108410 0x7f81ac1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 -- 172.21.15.3:0/1186449572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac108410 msgr2=0x7f81ac1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac108410 0x7f81ac1a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 -- 172.21.15.3:0/1186449572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ac1023a0 msgr2=0x7f81ac19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ac1023a0 0x7f81ac19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 -- 172.21.15.3:0/1186449572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ac1a5f50 con 0x7f81ac102d60 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b19b4700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac108410 0x7f81ac1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81b09b2700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 0x7f81ac19aed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f819c00c9b0 tx=0x7f819c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c007b00 con 0x7f81ac102d60 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.192+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f819c007ca0 con 0x7f81ac102d60 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.193+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ac1a61e0 con 0x7f81ac102d60 2024-05-08T05:49:33.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.193+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819c018660 con 0x7f81ac102d60 2024-05-08T05:49:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.193+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ac1a6740 con 0x7f81ac102d60 2024-05-08T05:49:33.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.194+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ac06a190 con 0x7f81ac102d60 2024-05-08T05:49:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.195+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f819c00f450 con 0x7f81ac102d60 2024-05-08T05:49:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.196+0000 7f81a27fc700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f81980b2900 0x7f81980b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.196+0000 7f81b11b3700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f81980b2900 0x7f81980b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.196+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f819c0e1f70 con 0x7f81ac102d60 2024-05-08T05:49:33.197 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.197+0000 7f81b11b3700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f81980b2900 0x7f81980b4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009fd0 tx=0x7f81a800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:33.200 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.200+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f819c014030 con 0x7f81ac102d60 2024-05-08T05:49:33.459 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 podman[65283]: 2024-05-08 05:49:33.064970786 +0000 UTC m=+0.018003457 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:33.459 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 podman[65283]: 2024-05-08 05:49:33.182045804 +0000 UTC m=+0.135078502 container create 4bf57299307a8b0203715740d9d16acb1efb09f420127248eac48ad6a95ae899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:49:33.459 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 podman[65283]: 2024-05-08 05:49:33.459612768 +0000 UTC m=+0.412645423 container init 4bf57299307a8b0203715740d9d16acb1efb09f420127248eac48ad6a95ae899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:49:33.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.474+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81ac02e510 con 0x7f81ac102d60 2024-05-08T05:49:33.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.474+0000 7f81a27fc700 1 -- 172.21.15.3:0/1186449572 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7171 (secure 0 0 0) 0x7f819c0aa240 con 0x7f81ac102d60 2024-05-08T05:49:33.475 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:33.475 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f81980b2900 msgr2=0x7f81980b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f81980b2900 0x7f81980b4dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009fd0 tx=0x7f81a800b7b0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 msgr2=0x7f81ac19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 0x7f81ac19aed0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f819c00c9b0 tx=0x7f819c00ccc0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 shutdown_connections 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ac108410 0x7f81ac1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81ac1023a0 0x7f81ac19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f81980b2900 0x7f81980b4dc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 --2- 172.21.15.3:0/1186449572 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81ac102d60 0x7f81ac19aed0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 >> 172.21.15.3:0/1186449572 conn(0x7f81ac079350 msgr2=0x7f81ac101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 shutdown_connections 2024-05-08T05:49:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:33.477+0000 7f81b3417700 1 -- 172.21.15.3:0/1186449572 wait complete. 2024-05-08T05:49:33.478 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 9 2024-05-08T05:49:33.719 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 podman[65283]: 2024-05-08 05:49:33.462494809 +0000 UTC m=+0.415527472 container start 4bf57299307a8b0203715740d9d16acb1efb09f420127248eac48ad6a95ae899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-08T05:49:33.720 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mds[65302]: -- 172.21.15.104:0/2780511720 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560d79d8fa40 con 0x560d79d63000 2024-05-08T05:49:33.720 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: starting mds.i at 2024-05-08T05:49:33.720 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 bash[65283]: 4bf57299307a8b0203715740d9d16acb1efb09f420127248eac48ad6a95ae899 2024-05-08T05:49:33.720 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:49:33 smithi104 systemd[1]: Started Ceph mds.i for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:33.851 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1186449572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:33.851 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:33.851 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:33.851 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:33.851 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:33.851 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:33.852 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1186449572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:34.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:34.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:34.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:34.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:34.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:34.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1186449572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:34.895 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:34 smithi104 ceph-mon[34559]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:34.896 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:34 smithi104 ceph-mon[34559]: Deploying daemon mds.g on smithi003 2024-05-08T05:49:34.896 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:34 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] up:boot 2024-05-08T05:49:34.896 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:34 smithi104 ceph-mon[34559]: fsmap 9 up:standby 2024-05-08T05:49:34.896 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:49:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:34 smithi003 ceph-mon[30137]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:34 smithi003 ceph-mon[30137]: Deploying daemon mds.g on smithi003 2024-05-08T05:49:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:34 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] up:boot 2024-05-08T05:49:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:34 smithi003 ceph-mon[30137]: fsmap 9 up:standby 2024-05-08T05:49:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:49:35.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:34 smithi089 ceph-mon[34495]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:35.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:34 smithi089 ceph-mon[34495]: Deploying daemon mds.g on smithi003 2024-05-08T05:49:35.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:34 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] up:boot 2024-05-08T05:49:35.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:34 smithi089 ceph-mon[34495]: fsmap 9 up:standby 2024-05-08T05:49:35.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:49:36.137 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:35 smithi003 systemd[1]: Starting Ceph mds.g for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:36.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:36.410 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 podman[72127]: 2024-05-08 05:49:36.039762559 +0000 UTC m=+0.018420664 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:36.410 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 podman[72127]: 2024-05-08 05:49:36.170619321 +0000 UTC m=+0.149277449 container create 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507) 2024-05-08T05:49:36.605 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:36.729 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 podman[72127]: 2024-05-08 05:49:36.415179226 +0000 UTC m=+0.393837320 container init 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-05-08T05:49:36.729 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 podman[72127]: 2024-05-08 05:49:36.418475576 +0000 UTC m=+0.397133665 container start 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T05:49:36.729 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mds[72177]: -- 172.21.15.3:0/3084715788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e25e843a40 con 0x55e25e816c00 2024-05-08T05:49:36.729 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g[72160]: starting mds.g at 2024-05-08T05:49:36.729 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 bash[72127]: 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da 2024-05-08T05:49:36.729 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 05:49:36 smithi003 systemd[1]: Started Ceph mds.g for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:37.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:37.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:37.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:37.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:36 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:36 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:37.026 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:36 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:37.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:37 smithi003 ceph-mon[30137]: Deploying daemon mds.a on smithi003 2024-05-08T05:49:37.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:37 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.3:6838/3875310227,v1:172.21.15.3:6839/3875310227] up:boot 2024-05-08T05:49:37.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:37 smithi003 ceph-mon[30137]: fsmap 10 up:standby 2024-05-08T05:49:37.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:37 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:49:38.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:37 smithi089 ceph-mon[34495]: Deploying daemon mds.a on smithi003 2024-05-08T05:49:38.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:37 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.3:6838/3875310227,v1:172.21.15.3:6839/3875310227] up:boot 2024-05-08T05:49:38.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:37 smithi089 ceph-mon[34495]: fsmap 10 up:standby 2024-05-08T05:49:38.015 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:37 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:49:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:37 smithi104 ceph-mon[34559]: Deploying daemon mds.a on smithi003 2024-05-08T05:49:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:37 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.3:6838/3875310227,v1:172.21.15.3:6839/3875310227] up:boot 2024-05-08T05:49:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:37 smithi104 ceph-mon[34559]: fsmap 10 up:standby 2024-05-08T05:49:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:37 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.523+0000 7fd3f4740700 1 -- 172.21.15.3:0/831587458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec108850 msgr2=0x7fd3ec1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.523+0000 7fd3f4740700 1 --2- 172.21.15.3:0/831587458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec108850 0x7fd3ec1103d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40099c0 tx=0x7fd3e4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 -- 172.21.15.3:0/831587458 shutdown_connections 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 --2- 172.21.15.3:0/831587458 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec108850 0x7fd3ec1103d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 --2- 172.21.15.3:0/831587458 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 0x7fd3ec103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 --2- 172.21.15.3:0/831587458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3ec104100 0x7fd3ec108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 -- 172.21.15.3:0/831587458 >> 172.21.15.3:0/831587458 conn(0x7fd3ec07a600 msgr2=0x7fd3ec07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 -- 172.21.15.3:0/831587458 shutdown_connections 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 -- 172.21.15.3:0/831587458 wait complete. 2024-05-08T05:49:38.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 Processor -- start 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.524+0000 7fd3f4740700 1 -- start start 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f4740700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 0x7fd3ec19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f4740700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 0x7fd3ec19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f4740700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3ec108850 0x7fd3ec199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f4740700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec113640 con 0x7fd3ec104100 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f4740700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1134c0 con 0x7fd3ec103750 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f4740700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec1137c0 con 0x7fd3ec108850 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 0x7fd3ec19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f24dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 0x7fd3ec19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 0x7fd3ec19fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60624/0 (socket says 172.21.15.3:60624) 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 -- 172.21.15.3:0/815685888 learned_addr learned my addr 172.21.15.3:0/815685888 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f2cdd700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3ec108850 0x7fd3ec199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:38.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 -- 172.21.15.3:0/815685888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3ec108850 msgr2=0x7fd3ec199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3ec108850 0x7fd3ec199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 -- 172.21.15.3:0/815685888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 msgr2=0x7fd3ec19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 0x7fd3ec19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 -- 172.21.15.3:0/815685888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec19a060 con 0x7fd3ec104100 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f24dc700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 0x7fd3ec19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3f1cdb700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 0x7fd3ec19fc80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c970 tx=0x7fd3dc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.525+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0078d0 con 0x7fd3ec104100 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.526+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec19a350 con 0x7fd3ec104100 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.526+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3dc018480 con 0x7fd3ec104100 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.526+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc00f6b0 con 0x7fd3ec104100 2024-05-08T05:49:38.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.526+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec1a6610 con 0x7fd3ec104100 2024-05-08T05:49:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.527+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ec06a190 con 0x7fd3ec104100 2024-05-08T05:49:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.528+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd3dc018930 con 0x7fd3ec104100 2024-05-08T05:49:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.529+0000 7fd3e37fe700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd3d80b2880 0x7fd3d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:38.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.529+0000 7fd3f24dc700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd3d80b2880 0x7fd3d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.529+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fd3dc0e21e0 con 0x7fd3ec104100 2024-05-08T05:49:38.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.530+0000 7fd3f24dc700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd3d80b2880 0x7fd3d80b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec19aa40 tx=0x7fd3e80091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:38.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.533+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3dc0aa430 con 0x7fd3ec104100 2024-05-08T05:49:38.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.830+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3ec19ae30 con 0x7fd3ec104100 2024-05-08T05:49:38.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.830+0000 7fd3e37fe700 1 -- 172.21.15.3:0/815685888 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fd3dc0a9d30 con 0x7fd3ec104100 2024-05-08T05:49:38.832 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:38.832 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-08T05:49:38.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd3d80b2880 msgr2=0x7fd3d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:38.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd3d80b2880 0x7fd3d80b4d40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec19aa40 tx=0x7fd3e80091b0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 msgr2=0x7fd3ec19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 0x7fd3ec19fc80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00c970 tx=0x7fd3dc00cc80 comp rx=0 tx=0).stop 2024-05-08T05:49:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 shutdown_connections 2024-05-08T05:49:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3ec108850 0x7fd3ec199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.836 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3ec103750 0x7fd3ec19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd3d80b2880 0x7fd3d80b4d40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 --2- 172.21.15.3:0/815685888 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3ec104100 0x7fd3ec19fc80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 >> 172.21.15.3:0/815685888 conn(0x7fd3ec07a600 msgr2=0x7fd3ec101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 shutdown_connections 2024-05-08T05:49:38.837 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:38.834+0000 7fd3f4740700 1 -- 172.21.15.3:0/815685888 wait complete. 2024-05-08T05:49:38.838 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 11 2024-05-08T05:49:39.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:38 smithi089 ceph-mon[34495]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:38 smithi104 ceph-mon[34559]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:39.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:38 smithi003 ceph-mon[30137]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:39.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:39 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/815685888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:39 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/815685888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:39 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/815685888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:40.660 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:40 smithi003 systemd[1]: Starting Ceph mds.a for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:41.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:40 smithi089 ceph-mon[34495]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:41.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:40 smithi104 ceph-mon[34559]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:41.049 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:40 smithi003 ceph-mon[30137]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:41.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:40 smithi003 podman[72591]: 2024-05-08 05:49:40.640876453 +0000 UTC m=+0.018748133 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:41.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:40 smithi003 podman[72591]: 2024-05-08 05:49:40.797505989 +0000 UTC m=+0.175377695 container create a199896766ac07535ba30e1255ff179c2d203780471834874a70a8cadad70d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:49:41.049 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:41 smithi003 podman[72591]: 2024-05-08 05:49:41.050066051 +0000 UTC m=+0.427937718 container init a199896766ac07535ba30e1255ff179c2d203780471834874a70a8cadad70d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:49:41.410 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:41 smithi003 podman[72591]: 2024-05-08 05:49:41.053243561 +0000 UTC m=+0.431115226 container start a199896766ac07535ba30e1255ff179c2d203780471834874a70a8cadad70d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:49:41.410 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:41 smithi003 ceph-mds[72610]: -- 172.21.15.3:0/3539422589 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557cb9b8da40 con 0x557cb9b61000 2024-05-08T05:49:41.410 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a[72606]: starting mds.a at 2024-05-08T05:49:41.410 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:41 smithi003 bash[72591]: a199896766ac07535ba30e1255ff179c2d203780471834874a70a8cadad70d0b 2024-05-08T05:49:41.410 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 05:49:41 smithi003 systemd[1]: Started Ceph mds.a for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:41.579 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:41.835 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:42.210 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: Deploying daemon mds.b on smithi089 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.3:6840/1028446466,v1:172.21.15.3:6841/1028446466] up:boot 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: fsmap 11 up:standby 2024-05-08T05:49:42.211 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:42 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: Deploying daemon mds.b on smithi089 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.3:6840/1028446466,v1:172.21.15.3:6841/1028446466] up:boot 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: fsmap 11 up:standby 2024-05-08T05:49:42.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:42 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: Deploying daemon mds.b on smithi089 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.3:6840/1028446466,v1:172.21.15.3:6841/1028446466] up:boot 2024-05-08T05:49:42.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: fsmap 11 up:standby 2024-05-08T05:49:42.522 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:42 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:49:43.513 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 systemd[1]: Starting Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:49:43.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 -- 172.21.15.3:0/2387345557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f810fef0 msgr2=0x7f25f81122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:43.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 --2- 172.21.15.3:0/2387345557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f810fef0 0x7f25f81122e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25f000b0a0 tx=0x7f25f000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 -- 172.21.15.3:0/2387345557 shutdown_connections 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 --2- 172.21.15.3:0/2387345557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f810fef0 0x7f25f81122e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 --2- 172.21.15.3:0/2387345557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806bc50 0x7f25f806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 --2- 172.21.15.3:0/2387345557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25f806c570 0x7f25f810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 -- 172.21.15.3:0/2387345557 >> 172.21.15.3:0/2387345557 conn(0x7f25f8079440 msgr2=0x7f25f8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 -- 172.21.15.3:0/2387345557 shutdown_connections 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.527+0000 7f26001c5700 1 -- 172.21.15.3:0/2387345557 wait complete. 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 Processor -- start 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 -- start start 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 0x7f25f819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806c570 0x7f25f819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:43.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25f810fef0 0x7f25f81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81158d0 con 0x7f25f806bc50 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8115750 con 0x7f25f810fef0 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f26001c5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8115a50 con 0x7f25f806c570 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f25fdf61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 0x7f25f819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f25fdf61700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 0x7f25f819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47012/0 (socket says 172.21.15.3:47012) 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f25fdf61700 1 -- 172.21.15.3:0/3121982165 learned_addr learned my addr 172.21.15.3:0/3121982165 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f25fe762700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25f810fef0 0x7f25f81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.528+0000 7f25fd760700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806c570 0x7f25f819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fdf61700 1 -- 172.21.15.3:0/3121982165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806c570 msgr2=0x7f25f819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fdf61700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806c570 0x7f25f819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fdf61700 1 -- 172.21.15.3:0/3121982165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25f810fef0 msgr2=0x7f25f81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fdf61700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25f810fef0 0x7f25f81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.531 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fdf61700 1 -- 172.21.15.3:0/3121982165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f81a0d40 con 0x7f25f806bc50 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fd760700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806c570 0x7f25f819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25fdf61700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 0x7f25f819bd80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c9b0 tx=0x7f25f400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4007b00 con 0x7f25f806bc50 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f81a1030 con 0x7f25f806bc50 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25f4007ca0 con 0x7f25f806bc50 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4018660 con 0x7f25f806bc50 2024-05-08T05:49:43.532 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.529+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81a88b0 con 0x7f25f806bc50 2024-05-08T05:49:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.530+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f806a190 con 0x7f25f806bc50 2024-05-08T05:49:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.531+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f25f400f450 con 0x7f25f806bc50 2024-05-08T05:49:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.532+0000 7f25eeffd700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25e40b2880 0x7f25e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.532+0000 7f25fd760700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25e40b2880 0x7f25e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.532+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f25f40e20b0 con 0x7f25f806bc50 2024-05-08T05:49:43.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.533+0000 7f25fd760700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25e40b2880 0x7f25e40b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25f8103850 tx=0x7f25e8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:43.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.536+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25f4014030 con 0x7f25f806bc50 2024-05-08T05:49:43.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.796+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25f802e510 con 0x7f25f806bc50 2024-05-08T05:49:43.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.797+0000 7f25eeffd700 1 -- 172.21.15.3:0/3121982165 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9647 (secure 0 0 0) 0x7f25f40aa260 con 0x7f25f806bc50 2024-05-08T05:49:43.797 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:43.797 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-08T05:49:43.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25e40b2880 msgr2=0x7f25e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25e40b2880 0x7f25e40b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25f8103850 tx=0x7f25e8005d80 comp rx=0 tx=0).stop 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 msgr2=0x7f25f819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 0x7f25f819bd80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f25f400c9b0 tx=0x7f25f400ccc0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 shutdown_connections 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25f810fef0 0x7f25f81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25e40b2880 0x7f25e40b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25f806bc50 0x7f25f819bd80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 --2- 172.21.15.3:0/3121982165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25f806c570 0x7f25f819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 >> 172.21.15.3:0/3121982165 conn(0x7f25f8079440 msgr2=0x7f25f8111340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 shutdown_connections 2024-05-08T05:49:43.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:43.799+0000 7f26001c5700 1 -- 172.21.15.3:0/3121982165 wait complete. 2024-05-08T05:49:43.801 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 12 2024-05-08T05:49:43.955 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 podman[65119]: 2024-05-08 05:49:43.558485763 +0000 UTC m=+0.018391065 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:49:43.955 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 podman[65119]: 2024-05-08 05:49:43.710579123 +0000 UTC m=+0.170484409 container create 6358f2ed239491acb6b6b9fe3be123703c3f44e9e01ea7f3b99c3c419c56c614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T05:49:43.955 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 podman[65119]: 2024-05-08 05:49:43.95575909 +0000 UTC m=+0.415664377 container init 6358f2ed239491acb6b6b9fe3be123703c3f44e9e01ea7f3b99c3c419c56c614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-08T05:49:44.219 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 podman[65119]: 2024-05-08 05:49:43.95853723 +0000 UTC m=+0.418442524 container start 6358f2ed239491acb6b6b9fe3be123703c3f44e9e01ea7f3b99c3c419c56c614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T05:49:44.220 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 ceph-mds[65138]: -- 172.21.15.89:0/1784639470 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b5b7f05a40 con 0x55b5b7ed8800 2024-05-08T05:49:44.220 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: starting mds.b at 2024-05-08T05:49:44.220 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:44 smithi089 bash[65119]: 6358f2ed239491acb6b6b9fe3be123703c3f44e9e01ea7f3b99c3c419c56c614 2024-05-08T05:49:44.220 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:49:44 smithi089 systemd[1]: Started Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:49:44.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3121982165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:44 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3121982165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:44 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3121982165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:44.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:44 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:45.375 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:45 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.89:6838/472688413,v1:172.21.15.89:6839/472688413] up:boot 2024-05-08T05:49:45.376 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:45 smithi089 ceph-mon[34495]: fsmap 12 up:standby 2024-05-08T05:49:45.376 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:49:45.376 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:45 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:45.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:45 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.89:6838/472688413,v1:172.21.15.89:6839/472688413] up:boot 2024-05-08T05:49:45.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:45 smithi003 ceph-mon[30137]: fsmap 12 up:standby 2024-05-08T05:49:45.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:49:45.449 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:45 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:45 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.89:6838/472688413,v1:172.21.15.89:6839/472688413] up:boot 2024-05-08T05:49:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:45 smithi104 ceph-mon[34559]: fsmap 12 up:standby 2024-05-08T05:49:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:49:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:45 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:46.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:49:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:46 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:46 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:46.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.526 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.527 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:46 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:46.760 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:47 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:47.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:47.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:47.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:47.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:47 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:47.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:47.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:47.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:47.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:47 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:49:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 -- 172.21.15.3:0/266504689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb144103540 msgr2=0x7fb1441039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 --2- 172.21.15.3:0/266504689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb144103540 0x7fb1441039c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1380099c0 tx=0x7fb138009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 -- 172.21.15.3:0/266504689 shutdown_connections 2024-05-08T05:49:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 --2- 172.21.15.3:0/266504689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb1441103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 --2- 172.21.15.3:0/266504689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1441095e0 0x7fb1441099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 --2- 172.21.15.3:0/266504689 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb144103540 0x7fb1441039c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 -- 172.21.15.3:0/266504689 >> 172.21.15.3:0/266504689 conn(0x7fb14407a650 msgr2=0x7fb14407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:48.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:48 smithi003 ceph-mon[30137]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:48.477 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:48 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 -- 172.21.15.3:0/266504689 shutdown_connections 2024-05-08T05:49:48.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.476+0000 7fb14b76d700 1 -- 172.21.15.3:0/266504689 wait complete. 2024-05-08T05:49:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.477+0000 7fb14b76d700 1 Processor -- start 2024-05-08T05:49:48.478 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.477+0000 7fb14b76d700 1 -- start start 2024-05-08T05:49:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb14b76d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb144103540 0x7fb14419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb14b76d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb14419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb14b76d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 0x7fb144199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb14b76d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb144113700 con 0x7fb1441095e0 2024-05-08T05:49:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb14b76d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb144113580 con 0x7fb144103f00 2024-05-08T05:49:48.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb14b76d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb144113880 con 0x7fb144103540 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb149d0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 0x7fb144199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb14419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb149509700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb144103540 0x7fb14419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb14419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:39188/0 (socket says 172.21.15.3:39188) 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb149d0a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 0x7fb144199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47026/0 (socket says 172.21.15.3:47026) 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 -- 172.21.15.3:0/4004413190 learned_addr learned my addr 172.21.15.3:0/4004413190 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 -- 172.21.15.3:0/4004413190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb144103540 msgr2=0x7fb14419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb144103540 0x7fb14419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 -- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 msgr2=0x7fb144199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 0x7fb144199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb148d08700 1 -- 172.21.15.3:0/4004413190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14419a0e0 con 0x7fb144103f00 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.478+0000 7fb149d0a700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 0x7fb144199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.479+0000 7fb148d08700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb14419fd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb138006010 tx=0x7fb1380045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.479+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138026070 con 0x7fb144103f00 2024-05-08T05:49:48.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.479+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14419a370 con 0x7fb144103f00 2024-05-08T05:49:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.479+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13800f460 con 0x7fb144103f00 2024-05-08T05:49:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.479+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138009e60 con 0x7fb144103f00 2024-05-08T05:49:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.479+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1441a6690 con 0x7fb144103f00 2024-05-08T05:49:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.480+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14410ba70 con 0x7fb144103f00 2024-05-08T05:49:48.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.481+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fb138029070 con 0x7fb144103f00 2024-05-08T05:49:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.482+0000 7fb1427fc700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb12c0b2880 0x7fb12c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.482+0000 7fb149509700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb12c0b2880 0x7fb12c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.483+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7fb1380e2610 con 0x7fb144103f00 2024-05-08T05:49:48.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.483+0000 7fb149509700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb12c0b2880 0x7fb12c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb14419aac0 tx=0x7fb13c009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:48.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.487+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb13805edf0 con 0x7fb144103f00 2024-05-08T05:49:48.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:48 smithi089 ceph-mon[34495]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:48.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:48 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:48.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:48 smithi104 ceph-mon[34559]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:48.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:48 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.780+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb14419aeb0 con 0x7fb144103f00 2024-05-08T05:49:48.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.780+0000 7fb1427fc700 1 -- 172.21.15.3:0/4004413190 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7fb138013070 con 0x7fb144103f00 2024-05-08T05:49:48.781 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:48.781 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-05-08T05:49:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb12c0b2880 msgr2=0x7fb12c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:48.783 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb12c0b2880 0x7fb12c0b4d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb14419aac0 tx=0x7fb13c009180 comp rx=0 tx=0).stop 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 msgr2=0x7fb14419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb14419fd00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb138006010 tx=0x7fb1380045e0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 shutdown_connections 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb12c0b2880 0x7fb12c0b4d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb1441095e0 0x7fb144199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb144103540 0x7fb14419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 --2- 172.21.15.3:0/4004413190 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb144103f00 0x7fb14419fd00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 >> 172.21.15.3:0/4004413190 conn(0x7fb14407a650 msgr2=0x7fb144102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 shutdown_connections 2024-05-08T05:49:48.784 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:48.783+0000 7fb14b76d700 1 -- 172.21.15.3:0/4004413190 wait complete. 2024-05-08T05:49:48.785 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 13 2024-05-08T05:49:49.562 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-08T05:49:49.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:49 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:49 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/4004413190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:49.940 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:49 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:49 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/4004413190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:49 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:49 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/4004413190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:49:50.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:50 smithi003 ceph-mon[30137]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:51.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:50 smithi089 ceph-mon[34495]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:51.048 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:50 smithi104 ceph-mon[34559]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 -- 172.21.15.3:0/3408255902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 msgr2=0x7f7958103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:51.677 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 --2- 172.21.15.3:0/3408255902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f7958103b30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f79480099c0 tx=0x7f7948009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 -- 172.21.15.3:0/3408255902 shutdown_connections 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 --2- 172.21.15.3:0/3408255902 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7958108850 0x7f79581103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 --2- 172.21.15.3:0/3408255902 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f7958103b30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 --2- 172.21.15.3:0/3408255902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958104100 0x7f7958108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 -- 172.21.15.3:0/3408255902 >> 172.21.15.3:0/3408255902 conn(0x7f795807a600 msgr2=0x7f795807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 -- 172.21.15.3:0/3408255902 shutdown_connections 2024-05-08T05:49:51.678 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.677+0000 7f795d45a700 1 -- 172.21.15.3:0/3408255902 wait complete. 2024-05-08T05:49:51.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 Processor -- start 2024-05-08T05:49:51.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 -- start start 2024-05-08T05:49:51.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f795819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:51.680 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7958104100 0x7f795819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958108850 0x7f79581999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7958113640 con 0x7f7958103750 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f79581134c0 con 0x7f7958104100 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f795d45a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79581137c0 con 0x7f7958108850 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f795819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f795819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47036/0 (socket says 172.21.15.3:47036) 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 -- 172.21.15.3:0/1379060428 learned_addr learned my addr 172.21.15.3:0/1379060428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f79567fc700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7958104100 0x7f795819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f79577fe700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958108850 0x7f79581999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 -- 172.21.15.3:0/1379060428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958108850 msgr2=0x7f79581999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958108850 0x7f79581999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 -- 172.21.15.3:0/1379060428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7958104100 msgr2=0x7f795819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7958104100 0x7f795819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.681 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f7956ffd700 1 -- 172.21.15.3:0/1379060428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795819a100 con 0x7f7958103750 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.678+0000 7f79577fe700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958108850 0x7f79581999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.679+0000 7f7956ffd700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f795819f790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f794800b580 tx=0x7f79480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.679+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948027070 con 0x7f7958103750 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.679+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7948003f20 con 0x7f7958103750 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.679+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795819a390 con 0x7f7958103750 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.679+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7948022d70 con 0x7f7958103750 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.679+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79580756d0 con 0x7f7958103750 2024-05-08T05:49:51.682 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.680+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795806a190 con 0x7f7958103750 2024-05-08T05:49:51.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.681+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7948022500 con 0x7f7958103750 2024-05-08T05:49:51.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.682+0000 7f794ffff700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f79440b2900 0x7f79440b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:51.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.682+0000 7f79567fc700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f79440b2900 0x7f79440b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:51.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.682+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f79480e2d50 con 0x7f7958103750 2024-05-08T05:49:51.683 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.683+0000 7f79567fc700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f79440b2900 0x7f79440b4dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7958103050 tx=0x7f794000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:51.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.687+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7948013070 con 0x7f7958103750 2024-05-08T05:49:51.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.927+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f795819ae80 con 0x7f7958103750 2024-05-08T05:49:51.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.927+0000 7f794ffff700 1 -- 172.21.15.3:0/1379060428 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) ==== 84+0+65 (secure 0 0 0) 0x7f79480aafa0 con 0x7f7958103750 2024-05-08T05:49:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:49:51.928 INFO:teuthology.orchestra.run.smithi003.stdout:[ 2024-05-08T05:49:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T05:49:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "poolnum": 1, 2024-05-08T05:49:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "poolname": ".mgr" 2024-05-08T05:49:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:49:51.928 INFO:teuthology.orchestra.run.smithi003.stdout:] 2024-05-08T05:49:51.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f79440b2900 msgr2=0x7f79440b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f79440b2900 0x7f79440b4dc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7958103050 tx=0x7f794000b3c0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 msgr2=0x7f795819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f795819f790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f794800b580 tx=0x7f79480047b0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 shutdown_connections 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7958108850 0x7f79581999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f79440b2900 0x7f79440b4dc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7958103750 0x7f795819f790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 --2- 172.21.15.3:0/1379060428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7958104100 0x7f795819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.929+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 >> 172.21.15.3:0/1379060428 conn(0x7f795807a600 msgr2=0x7f79581099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.930+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 shutdown_connections 2024-05-08T05:49:51.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:51.930+0000 7f795d45a700 1 -- 172.21.15.3:0/1379060428 wait complete. 2024-05-08T05:49:52.612 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:49:52.612 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:49:52.862 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1379060428' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:52.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:53.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1379060428' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:49:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:53.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:53.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:53.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1379060428' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:49:53.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:53.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:53.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:49:53.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:49:53.022 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:49:54.706 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:54 smithi003 ceph-mon[30137]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:54.711 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 -- 172.21.15.3:0/4101184173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102fd0 msgr2=0x7f5664107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 --2- 172.21.15.3:0/4101184173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102fd0 0x7f5664107020 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f56540099c0 tx=0x7f5654009cd0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 -- 172.21.15.3:0/4101184173 shutdown_connections 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 --2- 172.21.15.3:0/4101184173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5664107700 0x7f566410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 --2- 172.21.15.3:0/4101184173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664102620 0x7f5664102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 --2- 172.21.15.3:0/4101184173 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102fd0 0x7f5664107020 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 -- 172.21.15.3:0/4101184173 >> 172.21.15.3:0/4101184173 conn(0x7f5664079440 msgr2=0x7f5664079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 -- 172.21.15.3:0/4101184173 shutdown_connections 2024-05-08T05:49:54.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.711+0000 7f566b4dc700 1 -- 172.21.15.3:0/4101184173 wait complete. 2024-05-08T05:49:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 Processor -- start 2024-05-08T05:49:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 -- start start 2024-05-08T05:49:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 0x7f566419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:54.714 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5664102fd0 0x7f566419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664107700 0x7f5664198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5664112570 con 0x7f5664102620 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56641123f0 con 0x7f5664107700 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f566b4dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56641126f0 con 0x7f5664102fd0 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f5669278700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 0x7f566419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f5669278700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 0x7f566419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47370/0 (socket says 172.21.15.3:47370) 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.712+0000 7f5669278700 1 -- 172.21.15.3:0/2333790876 learned_addr learned my addr 172.21.15.3:0/2333790876 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669a79700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664107700 0x7f5664198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5668a77700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5664102fd0 0x7f566419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669278700 1 -- 172.21.15.3:0/2333790876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5664102fd0 msgr2=0x7f566419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669278700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5664102fd0 0x7f566419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669278700 1 -- 172.21.15.3:0/2333790876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664107700 msgr2=0x7f5664198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669278700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664107700 0x7f5664198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:54.715 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669278700 1 -- 172.21.15.3:0/2333790876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5664199010 con 0x7f5664102620 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669a79700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664107700 0x7f5664198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f5669278700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 0x7f566419e6b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f566000d950 tx=0x7f566000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660009ae0 con 0x7f5664102620 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5664199300 con 0x7f5664102620 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5660009c80 con 0x7f5664102620 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566001b7e0 con 0x7f5664102620 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.713+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566406bbd0 con 0x7f5664102620 2024-05-08T05:49:54.716 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.714+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5664103b00 con 0x7f5664102620 2024-05-08T05:49:54.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.716+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f5660010aa0 con 0x7f5664102620 2024-05-08T05:49:54.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.716+0000 7f565a7fc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f56500b2880 0x7f56500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:54.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.716+0000 7f5668a77700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f56500b2880 0x7f56500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:54.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.717+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7127+0+0 (secure 0 0 0) 0x7f56600e3de0 con 0x7f5664102620 2024-05-08T05:49:54.717 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.717+0000 7f5668a77700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f56500b2880 0x7f56500b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5654000c00 tx=0x7f565401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:54.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.721+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56600abfa0 con 0x7f5664102620 2024-05-08T05:49:54.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:54.935+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f566406c730 con 0x7f5664102620 2024-05-08T05:49:55.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:54 smithi089 ceph-mon[34495]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:55.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:54 smithi104 ceph-mon[34559]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:55.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.593+0000 7f565a7fc700 1 -- 172.21.15.3:0/2333790876 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) ==== 138+0+0 (secure 0 0 0) 0x7f56600ab8a0 con 0x7f5664102620 2024-05-08T05:49:55.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f56500b2880 msgr2=0x7f56500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:55.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f56500b2880 0x7f56500b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5654000c00 tx=0x7f565401a040 comp rx=0 tx=0).stop 2024-05-08T05:49:55.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 msgr2=0x7f566419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:55.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 0x7f566419e6b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f566000d950 tx=0x7f566000dd10 comp rx=0 tx=0).stop 2024-05-08T05:49:55.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 shutdown_connections 2024-05-08T05:49:55.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5664107700 0x7f5664198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:55.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f56500b2880 0x7f56500b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:55.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5664102620 0x7f566419e6b0 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f566000d950 tx=0x7f566000dd10 comp rx=0 tx=0).stop 2024-05-08T05:49:55.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 --2- 172.21.15.3:0/2333790876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5664102fd0 0x7f566419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:55.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 >> 172.21.15.3:0/2333790876 conn(0x7f5664079440 msgr2=0x7f5664100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:55.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 shutdown_connections 2024-05-08T05:49:55.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:55.597+0000 7f566b4dc700 1 -- 172.21.15.3:0/2333790876 wait complete. 2024-05-08T05:49:55.600 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' created 2024-05-08T05:49:55.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:55 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2333790876' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:49:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:55 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2333790876' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:49:56.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:55 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2333790876' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:49:56.235 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T05:49:56.492 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:56.871 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:56 smithi003 ceph-mon[30137]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:56.872 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:56 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2333790876' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:49:56.872 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:56 smithi003 ceph-mon[30137]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:49:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:56 smithi089 ceph-mon[34495]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:56 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2333790876' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:49:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:56 smithi089 ceph-mon[34495]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:49:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:56 smithi104 ceph-mon[34559]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:56 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2333790876' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:49:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:56 smithi104 ceph-mon[34559]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:49:57.905 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:57 smithi003 ceph-mon[30137]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:49:57.905 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:57 smithi003 ceph-mon[30137]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:49:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:57 smithi089 ceph-mon[34495]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:49:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:57 smithi089 ceph-mon[34495]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:49:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:57 smithi104 ceph-mon[34559]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:49:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:57 smithi104 ceph-mon[34559]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:49:58.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 -- 172.21.15.3:0/389825787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea88109500 msgr2=0x7fea881098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 --2- 172.21.15.3:0/389825787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea88109500 0x7fea881098e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea78009990 tx=0x7fea78009ca0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 -- 172.21.15.3:0/389825787 shutdown_connections 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 --2- 172.21.15.3:0/389825787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea8806c3d0 0x7fea88110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 --2- 172.21.15.3:0/389825787 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea88109500 0x7fea881098e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 --2- 172.21.15.3:0/389825787 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8806ba10 0x7fea8806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 -- 172.21.15.3:0/389825787 >> 172.21.15.3:0/389825787 conn(0x7fea88079310 msgr2=0x7fea88079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 -- 172.21.15.3:0/389825787 shutdown_connections 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 -- 172.21.15.3:0/389825787 wait complete. 2024-05-08T05:49:58.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 Processor -- start 2024-05-08T05:49:58.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.204+0000 7fea8e02d700 1 -- start start 2024-05-08T05:49:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea8e02d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 0x7fea8819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea8e02d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8806c3d0 0x7fea8819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:58.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea8e02d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea88109500 0x7fea88199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea8e02d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113610 con 0x7fea8806ba10 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea8e02d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113490 con 0x7fea8806c3d0 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea8e02d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113790 con 0x7fea88109500 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 0x7fea8819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 0x7fea8819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47386/0 (socket says 172.21.15.3:47386) 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 -- 172.21.15.3:0/1782441867 learned_addr learned my addr 172.21.15.3:0/1782441867 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea7effd700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8806c3d0 0x7fea8819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea87fff700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea88109500 0x7fea88199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 -- 172.21.15.3:0/1782441867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea88109500 msgr2=0x7fea88199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea88109500 0x7fea88199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 -- 172.21.15.3:0/1782441867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8806c3d0 msgr2=0x7fea8819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8806c3d0 0x7fea8819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea877fe700 1 -- 172.21.15.3:0/1782441867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819a010 con 0x7fea8806ba10 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.205+0000 7fea87fff700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea88109500 0x7fea88199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.206+0000 7fea877fe700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 0x7fea8819f6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea780057f0 tx=0x7fea78003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.206+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78027070 con 0x7fea8806ba10 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.206+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea780040a0 con 0x7fea8806ba10 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.206+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819a2a0 con 0x7fea8806ba10 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.206+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78022dc0 con 0x7fea8806ba10 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.206+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881a6510 con 0x7fea8806ba10 2024-05-08T05:49:58.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.207+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8810b9e0 con 0x7fea8806ba10 2024-05-08T05:49:58.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.208+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fea7802a070 con 0x7fea8806ba10 2024-05-08T05:49:58.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.209+0000 7fea857fa700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fea740b2880 0x7fea740b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:49:58.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.209+0000 7fea7effd700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fea740b2880 0x7fea740b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:49:58.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.209+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7497+0+0 (secure 0 0 0) 0x7fea780e2e20 con 0x7fea8806ba10 2024-05-08T05:49:58.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.210+0000 7fea7effd700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fea740b2880 0x7fea740b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea880798a0 tx=0x7fea7000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:49:58.214 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.214+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea78013070 con 0x7fea8806ba10 2024-05-08T05:49:58.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.448+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fea8819ad60 con 0x7fea8806ba10 2024-05-08T05:49:58.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.613+0000 7fea857fa700 1 -- 172.21.15.3:0/1782441867 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v75) ==== 171+0+0 (secure 0 0 0) 0x7fea780aae60 con 0x7fea8806ba10 2024-05-08T05:49:58.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.616+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fea740b2880 msgr2=0x7fea740b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:58.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.616+0000 7fea8e02d700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fea740b2880 0x7fea740b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fea880798a0 tx=0x7fea7000a380 comp rx=0 tx=0).stop 2024-05-08T05:49:58.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.616+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 msgr2=0x7fea8819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:49:58.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.616+0000 7fea8e02d700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 0x7fea8819f6b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea780057f0 tx=0x7fea78003680 comp rx=0 tx=0).stop 2024-05-08T05:49:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.617+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 shutdown_connections 2024-05-08T05:49:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.617+0000 7fea8e02d700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea88109500 0x7fea88199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.617+0000 7fea8e02d700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fea740b2880 0x7fea740b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.617+0000 7fea8e02d700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fea8806ba10 0x7fea8819f6b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.617+0000 7fea8e02d700 1 --2- 172.21.15.3:0/1782441867 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fea8806c3d0 0x7fea8819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:49:58.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.617+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 >> 172.21.15.3:0/1782441867 conn(0x7fea88079310 msgr2=0x7fea88100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:49:58.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.618+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 shutdown_connections 2024-05-08T05:49:58.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:49:58.618+0000 7fea8e02d700 1 -- 172.21.15.3:0/1782441867 wait complete. 2024-05-08T05:49:58.619 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' created 2024-05-08T05:49:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:58 smithi003 ceph-mon[30137]: pgmap v285: 33 pgs: 6 creating+activating, 3 active+clean, 24 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:58 smithi003 ceph-mon[30137]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:49:58.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:58 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1782441867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T05:49:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:58 smithi089 ceph-mon[34495]: pgmap v285: 33 pgs: 6 creating+activating, 3 active+clean, 24 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:58 smithi089 ceph-mon[34495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:49:59.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:58 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1782441867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T05:49:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:58 smithi104 ceph-mon[34559]: pgmap v285: 33 pgs: 6 creating+activating, 3 active+clean, 24 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:49:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:58 smithi104 ceph-mon[34559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:49:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:58 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1782441867' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T05:49:59.487 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:49:59.753 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:49:59.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:59 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1782441867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T05:49:59.875 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:49:59 smithi003 ceph-mon[30137]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:50:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:59 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1782441867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T05:50:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:49:59 smithi089 ceph-mon[34495]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:50:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:59 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1782441867' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T05:50:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:49:59 smithi104 ceph-mon[34559]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:50:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: pgmap v288: 97 pgs: 8 creating+peering, 6 creating+activating, 15 active+clean, 68 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:50:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-05-08T05:50:00.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-05-08T05:50:00.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: application not enabled on pool 'cephfs_metadata' 2024-05-08T05:50:00.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: application not enabled on pool 'cephfs_data' 2024-05-08T05:50:00.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:00 smithi003 ceph-mon[30137]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-05-08T05:50:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: pgmap v288: 97 pgs: 8 creating+peering, 6 creating+activating, 15 active+clean, 68 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:50:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-05-08T05:50:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-05-08T05:50:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: application not enabled on pool 'cephfs_metadata' 2024-05-08T05:50:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: application not enabled on pool 'cephfs_data' 2024-05-08T05:50:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:00 smithi089 ceph-mon[34495]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: pgmap v288: 97 pgs: 8 creating+peering, 6 creating+activating, 15 active+clean, 68 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: application not enabled on pool 'cephfs_metadata' 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: application not enabled on pool 'cephfs_data' 2024-05-08T05:50:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:00 smithi104 ceph-mon[34559]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-05-08T05:50:01.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3574157571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b881023a0 msgr2=0x7f7b88102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3574157571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b881023a0 0x7f7b88102820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009990 tx=0x7f7b7c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3574157571 shutdown_connections 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3574157571 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b88102d60 0x7f7b8810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3574157571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b88108410 0x7f7b881087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3574157571 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b881023a0 0x7f7b88102820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3574157571 >> 172.21.15.3:0/3574157571 conn(0x7f7b88079350 msgr2=0x7f7b88079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3574157571 shutdown_connections 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.536+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3574157571 wait complete. 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 Processor -- start 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 -- start start 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b881023a0 0x7f7b8819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b88102d60 0x7f7b8819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 0x7f7b88199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7b88112550 con 0x7f7b88108410 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881123d0 con 0x7f7b88102d60 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8ec17700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b881126d0 con 0x7f7b881023a0 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.537+0000 7f7b8d1b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 0x7f7b88199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 0x7f7b88199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47406/0 (socket says 172.21.15.3:47406) 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 -- 172.21.15.3:0/3109746178 learned_addr learned my addr 172.21.15.3:0/3109746178 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b87fff700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b88102d60 0x7f7b8819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8c9b3700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b881023a0 0x7f7b8819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 -- 172.21.15.3:0/3109746178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b881023a0 msgr2=0x7f7b8819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b881023a0 0x7f7b8819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 -- 172.21.15.3:0/3109746178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b88102d60 msgr2=0x7f7b8819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b88102d60 0x7f7b8819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:01.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 -- 172.21.15.3:0/3109746178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8819a160 con 0x7f7b88108410 2024-05-08T05:50:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8d1b4700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 0x7f7b88199a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000ada0 tx=0x7f7b8000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80017020 con 0x7f7b88108410 2024-05-08T05:50:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8819a450 con 0x7f7b88108410 2024-05-08T05:50:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8000ce30 con 0x7f7b88108410 2024-05-08T05:50:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b800054a0 con 0x7f7b88108410 2024-05-08T05:50:01.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.538+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b881a66c0 con 0x7f7b88108410 2024-05-08T05:50:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.540+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8806a190 con 0x7f7b88108410 2024-05-08T05:50:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.541+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7b8000c980 con 0x7f7b88108410 2024-05-08T05:50:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.542+0000 7f7b85ffb700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7b700b2900 0x7f7b700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.542+0000 7f7b8c9b3700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7b700b2900 0x7f7b700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.542+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7879+0+0 (secure 0 0 0) 0x7f7b800e1840 con 0x7f7b88108410 2024-05-08T05:50:01.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.542+0000 7f7b8c9b3700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7b700b2900 0x7f7b700b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b78005fd0 tx=0x7f7b78005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:01.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.546+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7b80021020 con 0x7f7b88108410 2024-05-08T05:50:01.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:01.807+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7b881a6960 con 0x7f7b88108410 2024-05-08T05:50:01.808 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:01 smithi003 ceph-mon[30137]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:50:02.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:01 smithi089 ceph-mon[34495]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:50:02.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:01 smithi104 ceph-mon[34559]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:50:02.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.637+0000 7f7b85ffb700 1 -- 172.21.15.3:0/3109746178 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f7b800a9700 con 0x7f7b88108410 2024-05-08T05:50:02.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7b700b2900 msgr2=0x7f7b700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:02.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7b700b2900 0x7f7b700b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b78005fd0 tx=0x7f7b78005d90 comp rx=0 tx=0).stop 2024-05-08T05:50:02.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 msgr2=0x7f7b88199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:02.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 0x7f7b88199a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000ada0 tx=0x7f7b8000c330 comp rx=0 tx=0).stop 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 shutdown_connections 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7b700b2900 0x7f7b700b4dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7b88108410 0x7f7b88199a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b881023a0 0x7f7b8819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 --2- 172.21.15.3:0/3109746178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7b88102d60 0x7f7b8819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.640+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 >> 172.21.15.3:0/3109746178 conn(0x7f7b88079350 msgr2=0x7f7b88101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:02.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.641+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 shutdown_connections 2024-05-08T05:50:02.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:02.641+0000 7f7b8ec17700 1 -- 172.21.15.3:0/3109746178 wait complete. 2024-05-08T05:50:02.643 INFO:teuthology.orchestra.run.smithi003.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T05:50:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:02 smithi003 ceph-mon[30137]: pgmap v291: 97 pgs: 29 creating+peering, 6 creating+activating, 31 active+clean, 31 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:02 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3109746178' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:50:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:02 smithi003 ceph-mon[30137]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-08T05:50:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:02 smithi089 ceph-mon[34495]: pgmap v291: 97 pgs: 29 creating+peering, 6 creating+activating, 31 active+clean, 31 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:02 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3109746178' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:50:03.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:02 smithi089 ceph-mon[34495]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-08T05:50:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:02 smithi104 ceph-mon[34559]: pgmap v291: 97 pgs: 29 creating+peering, 6 creating+activating, 31 active+clean, 31 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:02 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3109746178' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:50:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:02 smithi104 ceph-mon[34559]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-05-08T05:50:03.450 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T05:50:03.450 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T05:50:03.712 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:03.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:03 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3109746178' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:50:03.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:03 smithi003 ceph-mon[30137]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:50:03.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:03 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-08T05:50:03.835 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:03 smithi003 ceph-mon[30137]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-08T05:50:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:03 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3109746178' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:50:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:03 smithi089 ceph-mon[34495]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:50:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:03 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-08T05:50:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:03 smithi089 ceph-mon[34495]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-08T05:50:04.021 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 05:50:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: 2024-05-08T05:50:03.727+0000 7f388edec700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:50:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:03 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3109746178' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:50:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:03 smithi104 ceph-mon[34559]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:50:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:03 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-05-08T05:50:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:03 smithi104 ceph-mon[34559]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-05-08T05:50:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:04 smithi003 ceph-mon[30137]: pgmap v293: 97 pgs: 29 creating+peering, 64 active+clean, 4 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 0 op/s 2024-05-08T05:50:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:04 smithi003 ceph-mon[30137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:50:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:04 smithi003 ceph-mon[30137]: Cluster is now healthy 2024-05-08T05:50:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:04 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] up:active 2024-05-08T05:50:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:04 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:04 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:05.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:04 smithi089 ceph-mon[34495]: pgmap v293: 97 pgs: 29 creating+peering, 64 active+clean, 4 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 0 op/s 2024-05-08T05:50:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:04 smithi089 ceph-mon[34495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:50:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:04 smithi089 ceph-mon[34495]: Cluster is now healthy 2024-05-08T05:50:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:04 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] up:active 2024-05-08T05:50:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:04 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:05.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:04 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:04 smithi104 ceph-mon[34559]: pgmap v293: 97 pgs: 29 creating+peering, 64 active+clean, 4 unknown; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 222 B/s wr, 0 op/s 2024-05-08T05:50:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:04 smithi104 ceph-mon[34559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:50:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:04 smithi104 ceph-mon[34559]: Cluster is now healthy 2024-05-08T05:50:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:04 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] up:active 2024-05-08T05:50:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:04 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:04 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 -- 172.21.15.3:0/3689880246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f81020c0 msgr2=0x7fe9f81024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 --2- 172.21.15.3:0/3689880246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f81020c0 0x7fe9f81024a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8009990 tx=0x7fe9e8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 -- 172.21.15.3:0/3689880246 shutdown_connections 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 --2- 172.21.15.3:0/3689880246 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f8106f60 0x7fe9f810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 --2- 172.21.15.3:0/3689880246 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f81020c0 0x7fe9f81024a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 --2- 172.21.15.3:0/3689880246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9f8102a70 0x7fe9f8106830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:05.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 -- 172.21.15.3:0/3689880246 >> 172.21.15.3:0/3689880246 conn(0x7fe9f80776c0 msgr2=0x7fe9f8077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 -- 172.21.15.3:0/3689880246 shutdown_connections 2024-05-08T05:50:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 -- 172.21.15.3:0/3689880246 wait complete. 2024-05-08T05:50:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.538+0000 7fe9fe148700 1 Processor -- start 2024-05-08T05:50:05.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 -- start start 2024-05-08T05:50:05.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f81020c0 0x7fe9f819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 0x7fe9f819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9f8106f60 0x7fe9f81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8111510 con 0x7fe9f8102a70 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8111390 con 0x7fe9f81020c0 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9fe148700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f8111690 con 0x7fe9f8106f60 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 0x7fe9f819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 0x7fe9f819c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54656/0 (socket says 172.21.15.3:54656) 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 -- 172.21.15.3:0/107875223 learned_addr learned my addr 172.21.15.3:0/107875223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:05.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f77fe700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f81020c0 0x7fe9f819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f7fff700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9f8106f60 0x7fe9f81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 -- 172.21.15.3:0/107875223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9f8106f60 msgr2=0x7fe9f81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9f8106f60 0x7fe9f81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 -- 172.21.15.3:0/107875223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f81020c0 msgr2=0x7fe9f819bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f81020c0 0x7fe9f819bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.539+0000 7fe9f6ffd700 1 -- 172.21.15.3:0/107875223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f81a0d50 con 0x7fe9f8102a70 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.540+0000 7fe9f6ffd700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 0x7fe9f819c280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00dbc0 tx=0x7fe9ec00ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.540+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00ac10 con 0x7fe9f8102a70 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.540+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f81a1040 con 0x7fe9f8102a70 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.540+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9ec00adb0 con 0x7fe9f8102a70 2024-05-08T05:50:05.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.540+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec00e450 con 0x7fe9f8102a70 2024-05-08T05:50:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.540+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81a88c0 con 0x7fe9f8102a70 2024-05-08T05:50:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.542+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f806a190 con 0x7fe9f8102a70 2024-05-08T05:50:05.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.543+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe9ec004750 con 0x7fe9f8102a70 2024-05-08T05:50:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.543+0000 7fe9f4ff9700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9e00b2900 0x7fe9e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.543+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7979+0+0 (secure 0 0 0) 0x7fe9ec0e1dc0 con 0x7fe9f8102a70 2024-05-08T05:50:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.543+0000 7fe9f77fe700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9e00b2900 0x7fe9e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:05.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.544+0000 7fe9f77fe700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9e00b2900 0x7fe9e00b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800b5c0 tx=0x7fe9e8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:05.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.548+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9ec0a9c30 con 0x7fe9f8102a70 2024-05-08T05:50:05.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:05.772+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe9f81a8af0 con 0x7fe9f8102a70 2024-05-08T05:50:06.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.655+0000 7fe9f4ff9700 1 -- 172.21.15.3:0/107875223 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v79) ==== 157+0+0 (secure 0 0 0) 0x7fe9ec0a9530 con 0x7fe9f8102a70 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9e00b2900 msgr2=0x7fe9e00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9e00b2900 0x7fe9e00b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800b5c0 tx=0x7fe9e8005960 comp rx=0 tx=0).stop 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 msgr2=0x7fe9f819c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 0x7fe9f819c280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00dbc0 tx=0x7fe9ec00ded0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 shutdown_connections 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9f8106f60 0x7fe9f81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe9f81020c0 0x7fe9f819bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe9e00b2900 0x7fe9e00b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 --2- 172.21.15.3:0/107875223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe9f8102a70 0x7fe9f819c280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 >> 172.21.15.3:0/107875223 conn(0x7fe9f80776c0 msgr2=0x7fe9f81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 shutdown_connections 2024-05-08T05:50:06.659 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:06.658+0000 7fe9fe148700 1 -- 172.21.15.3:0/107875223 wait complete. 2024-05-08T05:50:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:06 smithi003 ceph-mon[30137]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-05-08T05:50:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:06 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/107875223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T05:50:07.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:06 smithi089 ceph-mon[34495]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-05-08T05:50:07.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:06 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/107875223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T05:50:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:06 smithi104 ceph-mon[34559]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-05-08T05:50:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:06 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/107875223' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T05:50:07.447 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:50:07.710 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:07.993 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:07 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/107875223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T05:50:07.993 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:07 smithi003 ceph-mon[30137]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:50:08.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:07 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/107875223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T05:50:08.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:07 smithi089 ceph-mon[34495]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:50:08.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:07 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/107875223' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T05:50:08.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:07 smithi104 ceph-mon[34559]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:50:08.997 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:08 smithi003 ceph-mon[30137]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-08T05:50:09.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:08 smithi089 ceph-mon[34495]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-08T05:50:09.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:08 smithi104 ceph-mon[34559]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-05-08T05:50:09.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 -- 172.21.15.3:0/649356362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0101ef0 msgr2=0x7f10b0102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:09.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/649356362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0101ef0 0x7f10b0102370 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f10a40099c0 tx=0x7f10a4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 -- 172.21.15.3:0/649356362 shutdown_connections 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/649356362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b01028b0 0x7f10b01103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/649356362 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0109620 0x7f10b0109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/649356362 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0101ef0 0x7f10b0102370 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 -- 172.21.15.3:0/649356362 >> 172.21.15.3:0/649356362 conn(0x7f10b0077640 msgr2=0x7f10b0077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 -- 172.21.15.3:0/649356362 shutdown_connections 2024-05-08T05:50:09.490 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 -- 172.21.15.3:0/649356362 wait complete. 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.478+0000 7f10b5c7e700 1 Processor -- start 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 -- start start 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0101ef0 0x7f10b019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b01028b0 0x7f10b019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 0x7f10b01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0113720 con 0x7f10b0109620 2024-05-08T05:50:09.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01135a0 con 0x7f10b01028b0 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10b5c7e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01138a0 con 0x7f10b0101ef0 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 0x7f10b01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 0x7f10b01998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:54672/0 (socket says 172.21.15.3:54672) 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 -- 172.21.15.3:0/3472897643 learned_addr learned my addr 172.21.15.3:0/3472897643 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10aeffd700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b01028b0 0x7f10b019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10af7fe700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0101ef0 0x7f10b019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 -- 172.21.15.3:0/3472897643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0101ef0 msgr2=0x7f10b019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:09.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0101ef0 0x7f10b019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 -- 172.21.15.3:0/3472897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b01028b0 msgr2=0x7f10b019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b01028b0 0x7f10b019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10affff700 1 -- 172.21.15.3:0/3472897643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b019a000 con 0x7f10b0109620 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.479+0000 7f10af7fe700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0101ef0 0x7f10b019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.480+0000 7f10affff700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 0x7f10b01998a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10a800d980 tx=0x7f10a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.480+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8016070 con 0x7f10b0109620 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.480+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8004830 con 0x7f10b0109620 2024-05-08T05:50:09.493 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.480+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b019a2f0 con 0x7f10b0109620 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.480+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a80054b0 con 0x7f10b0109620 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.480+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a65b0 con 0x7f10b0109620 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.481+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b010ba00 con 0x7f10b0109620 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.482+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f10a80049e0 con 0x7f10b0109620 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.483+0000 7f10acff9700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f10980b2930 0x7f10980b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.483+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8277+0+0 (secure 0 0 0) 0x7f10a80e2620 con 0x7f10b0109620 2024-05-08T05:50:09.494 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.483+0000 7f10af7fe700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f10980b2930 0x7f10980b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:09.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.484+0000 7f10af7fe700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f10980b2930 0x7f10980b4df0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10b019a9e0 tx=0x7f10a0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:09.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.486+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10a801f020 con 0x7f10b0109620 2024-05-08T05:50:09.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:09.705+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f10b019add0 con 0x7f10b0109620 2024-05-08T05:50:11.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:10 smithi089 ceph-mon[34495]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:50:11.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:10 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3472897643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:50:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:10 smithi104 ceph-mon[34559]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:50:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:10 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3472897643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:50:11.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:10 smithi003 ceph-mon[30137]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:50:11.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:10 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3472897643' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:50:11.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.683+0000 7f10acff9700 1 -- 172.21.15.3:0/3472897643 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v81) ==== 228+0+0 (secure 0 0 0) 0x7f10a80aa310 con 0x7f10b0109620 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f10980b2930 msgr2=0x7f10980b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f10980b2930 0x7f10980b4df0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10b019a9e0 tx=0x7f10a0009040 comp rx=0 tx=0).stop 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 msgr2=0x7f10b01998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 0x7f10b01998a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10a800d980 tx=0x7f10a800dc90 comp rx=0 tx=0).stop 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 shutdown_connections 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f10980b2930 0x7f10980b4df0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f10b0109620 0x7f10b01998a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10b0101ef0 0x7f10b019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 --2- 172.21.15.3:0/3472897643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f10b01028b0 0x7f10b019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 >> 172.21.15.3:0/3472897643 conn(0x7f10b0077640 msgr2=0x7f10b0100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:11.689 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 shutdown_connections 2024-05-08T05:50:11.690 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:11.688+0000 7f10b5c7e700 1 -- 172.21.15.3:0/3472897643 wait complete. 2024-05-08T05:50:11.690 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' created 2024-05-08T05:50:12.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:11 smithi089 ceph-mon[34495]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:50:12.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:11 smithi003 ceph-mon[30137]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:50:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:11 smithi104 ceph-mon[34559]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:50:12.452 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T05:50:12.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:12 smithi003 ceph-mon[30137]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T05:50:12.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:12 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3472897643' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:50:12.700 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:12 smithi003 ceph-mon[30137]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:50:12.730 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:12 smithi089 ceph-mon[34495]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T05:50:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:12 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3472897643' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:50:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:12 smithi089 ceph-mon[34495]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:50:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:12 smithi104 ceph-mon[34559]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-05-08T05:50:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:12 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3472897643' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:50:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:12 smithi104 ceph-mon[34559]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:50:13.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:13 smithi003 ceph-mon[30137]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:50:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:13 smithi089 ceph-mon[34495]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:50:14.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:13 smithi104 ceph-mon[34559]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:50:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/3805057775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38104100 msgr2=0x7f7d38108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/3805057775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38104100 0x7f7d38108120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d240099c0 tx=0x7f7d24009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/3805057775 shutdown_connections 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/3805057775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38108850 0x7f7d381103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/3805057775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 0x7f7d38103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/3805057775 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38104100 0x7f7d38108120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/3805057775 >> 172.21.15.3:0/3805057775 conn(0x7f7d3807a600 msgr2=0x7f7d3807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/3805057775 shutdown_connections 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.627+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/3805057775 wait complete. 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 Processor -- start 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 -- start start 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 0x7f7d3810f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38104100 0x7f7d3810fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 0x7f7d381a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.704 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38113610 con 0x7f7d38108850 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38113490 con 0x7f7d38104100 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ecd5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d38113790 con 0x7f7d38103750 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 0x7f7d381a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 0x7f7d381a4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44876/0 (socket says 172.21.15.3:44876) 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 -- 172.21.15.3:0/1680997122 learned_addr learned my addr 172.21.15.3:0/1680997122 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ca71700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 0x7f7d3810f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d37fff700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38104100 0x7f7d3810fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 -- 172.21.15.3:0/1680997122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 msgr2=0x7f7d3810f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 0x7f7d3810f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 -- 172.21.15.3:0/1680997122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38104100 msgr2=0x7f7d3810fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:14.705 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38104100 0x7f7d3810fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3d272700 1 -- 172.21.15.3:0/1680997122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d381a5030 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.628+0000 7f7d3ca71700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 0x7f7d3810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.629+0000 7f7d3d272700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 0x7f7d381a4920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d30007d70 tx=0x7f7d3000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.629+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30016020 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.629+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d3000ad40 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.629+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d381a5320 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.629+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d300115e0 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.629+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d381acbd0 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.630+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d3806a190 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.631+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7d30019020 con 0x7f7d38108850 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.632+0000 7f7d35ffb700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7d280b2900 0x7f7d280b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:14.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.632+0000 7f7d3ca71700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7d280b2900 0x7f7d280b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.632+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8770+0+0 (secure 0 0 0) 0x7f7d300e2890 con 0x7f7d38108850 2024-05-08T05:50:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.633+0000 7f7d3ca71700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7d280b2900 0x7f7d280b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005950 tx=0x7f7d2c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:14.707 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.636+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d30020020 con 0x7f7d38108850 2024-05-08T05:50:14.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:14.852+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f7d381ace70 con 0x7f7d38108850 2024-05-08T05:50:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:14 smithi003 ceph-mon[30137]: pgmap v302: 129 pgs: 13 creating+peering, 14 unknown, 102 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T05:50:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:14 smithi003 ceph-mon[30137]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:50:14.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:14 smithi003 ceph-mon[30137]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:50:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:14 smithi089 ceph-mon[34495]: pgmap v302: 129 pgs: 13 creating+peering, 14 unknown, 102 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T05:50:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:14 smithi089 ceph-mon[34495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:50:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:14 smithi089 ceph-mon[34495]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:50:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:14 smithi104 ceph-mon[34559]: pgmap v302: 129 pgs: 13 creating+peering, 14 unknown, 102 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-05-08T05:50:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:14 smithi104 ceph-mon[34559]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:50:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:14 smithi104 ceph-mon[34559]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:50:15.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.707+0000 7f7d35ffb700 1 -- 172.21.15.3:0/1680997122 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v84) ==== 169+0+0 (secure 0 0 0) 0x7f7d300aa330 con 0x7f7d38108850 2024-05-08T05:50:15.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7d280b2900 msgr2=0x7f7d280b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:15.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7d280b2900 0x7f7d280b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c005950 tx=0x7f7d2c0058e0 comp rx=0 tx=0).stop 2024-05-08T05:50:15.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 msgr2=0x7f7d381a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:15.712 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 0x7f7d381a4920 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7d30007d70 tx=0x7f7d3000a340 comp rx=0 tx=0).stop 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 shutdown_connections 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7d280b2900 0x7f7d280b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d38108850 0x7f7d381a4920 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d38103750 0x7f7d3810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 --2- 172.21.15.3:0/1680997122 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d38104100 0x7f7d3810fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.712+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 >> 172.21.15.3:0/1680997122 conn(0x7f7d3807a600 msgr2=0x7f7d38109a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.713+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 shutdown_connections 2024-05-08T05:50:15.713 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:15.713+0000 7f7d3ecd5700 1 -- 172.21.15.3:0/1680997122 wait complete. 2024-05-08T05:50:15.714 INFO:teuthology.orchestra.run.smithi003.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T05:50:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:15 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1680997122' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T05:50:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:15 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1680997122' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T05:50:16.090 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:15 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1680997122' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T05:50:16.526 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T05:50:16.787 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:16.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:16 smithi003 ceph-mon[30137]: pgmap v304: 129 pgs: 13 creating+peering, 4 unknown, 112 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:16 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1680997122' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T05:50:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:16 smithi003 ceph-mon[30137]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:50:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:16 smithi089 ceph-mon[34495]: pgmap v304: 129 pgs: 13 creating+peering, 4 unknown, 112 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:16 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1680997122' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T05:50:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:16 smithi089 ceph-mon[34495]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:50:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:16 smithi104 ceph-mon[34559]: pgmap v304: 129 pgs: 13 creating+peering, 4 unknown, 112 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:16 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1680997122' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T05:50:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:16 smithi104 ceph-mon[34559]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:50:18.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 -- 172.21.15.3:0/2154255219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2954108450 msgr2=0x7f2954108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2154255219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2954108450 0x7f2954108830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2948009990 tx=0x7f2948009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 -- 172.21.15.3:0/2154255219 shutdown_connections 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2154255219 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954102d90 0x7f295410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2154255219 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2954108450 0x7f2954108830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2154255219 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f29541023d0 0x7f2954102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 -- 172.21.15.3:0/2154255219 >> 172.21.15.3:0/2154255219 conn(0x7f2954078410 msgr2=0x7f2954078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 -- 172.21.15.3:0/2154255219 shutdown_connections 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 -- 172.21.15.3:0/2154255219 wait complete. 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 Processor -- start 2024-05-08T05:50:18.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.535+0000 7f295b1e6700 1 -- start start 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f295b1e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 0x7f295410fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f295b1e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2954102d90 0x7f29541100a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f295b1e6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954108450 0x7f29541105e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f295b1e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2954113740 con 0x7f29541023d0 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f295b1e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f29541135c0 con 0x7f2954102d90 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f295b1e6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f29541138c0 con 0x7f2954108450 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 0x7f295410fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 0x7f295410fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44896/0 (socket says 172.21.15.3:44896) 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 -- 172.21.15.3:0/2715176009 learned_addr learned my addr 172.21.15.3:0/2715176009 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2953fff700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2954102d90 0x7f29541100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:18.542 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2959783700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954108450 0x7f29541105e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 -- 172.21.15.3:0/2715176009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954108450 msgr2=0x7f29541105e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954108450 0x7f29541105e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 -- 172.21.15.3:0/2715176009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2954102d90 msgr2=0x7f29541100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2954102d90 0x7f29541100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 -- 172.21.15.3:0/2715176009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29541a8140 con 0x7f29541023d0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2959783700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954108450 0x7f29541105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2958f82700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 0x7f295410fb60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f29480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.536+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948027070 con 0x7f29541023d0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.537+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29541a83d0 con 0x7f29541023d0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.537+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2948004020 con 0x7f29541023d0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.537+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948022e00 con 0x7f29541023d0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.537+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a8880 con 0x7f29541023d0 2024-05-08T05:50:18.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.538+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295410a910 con 0x7f29541023d0 2024-05-08T05:50:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.539+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f294802a070 con 0x7f29541023d0 2024-05-08T05:50:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.540+0000 7f2951ffb700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f29440b2880 0x7f29440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.540+0000 7f2953fff700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f29440b2880 0x7f29440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.540+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f29480e3c40 con 0x7f29541023d0 2024-05-08T05:50:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.541+0000 7f2953fff700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f29440b2880 0x7f29440b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2954078a80 tx=0x7f294000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:18.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.543+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2948031070 con 0x7f29541023d0 2024-05-08T05:50:18.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:18.808+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2954111090 con 0x7f29541023d0 2024-05-08T05:50:18.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:18 smithi003 ceph-mon[30137]: pgmap v306: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:18 smithi089 ceph-mon[34495]: pgmap v306: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:19.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:18 smithi104 ceph-mon[34559]: pgmap v306: 129 pgs: 9 creating+peering, 120 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:19.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.725+0000 7f2951ffb700 1 -- 172.21.15.3:0/2715176009 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f2948013070 con 0x7f29541023d0 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f29440b2880 msgr2=0x7f29440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f29440b2880 0x7f29440b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2954078a80 tx=0x7f294000a300 comp rx=0 tx=0).stop 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 msgr2=0x7f295410fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 0x7f295410fb60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f29480047b0 comp rx=0 tx=0).stop 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 shutdown_connections 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2954108450 0x7f29541105e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f29440b2880 0x7f29440b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:19.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f29541023d0 0x7f295410fb60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 --2- 172.21.15.3:0/2715176009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2954102d90 0x7f29541100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 >> 172.21.15.3:0/2715176009 conn(0x7f2954078410 msgr2=0x7f2954101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.728+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 shutdown_connections 2024-05-08T05:50:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:19.729+0000 7f295b1e6700 1 -- 172.21.15.3:0/2715176009 wait complete. 2024-05-08T05:50:19.730 INFO:teuthology.orchestra.run.smithi003.stderr:added data pool 4 to fsmap 2024-05-08T05:50:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:19 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2715176009' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T05:50:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:19 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2715176009' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T05:50:20.063 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:19 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2715176009' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T05:50:20.555 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:20.818 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:20 smithi003 ceph-mon[30137]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:20 smithi003 ceph-mon[30137]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T05:50:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:20 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2715176009' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T05:50:20.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:20 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:21.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:20 smithi089 ceph-mon[34495]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:20 smithi089 ceph-mon[34495]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T05:50:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:20 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2715176009' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T05:50:21.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:20 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:20 smithi104 ceph-mon[34559]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:20 smithi104 ceph-mon[34559]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T05:50:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:20 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2715176009' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T05:50:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:20 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:50:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:21 smithi104 ceph-mon[34559]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:21 smithi104 ceph-mon[34559]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:50:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:21 smithi104 ceph-mon[34559]: Cluster is now healthy 2024-05-08T05:50:22.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:21 smithi003 ceph-mon[30137]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:22.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:21 smithi003 ceph-mon[30137]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:50:22.103 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:21 smithi003 ceph-mon[30137]: Cluster is now healthy 2024-05-08T05:50:22.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:21 smithi089 ceph-mon[34495]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:22.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:21 smithi089 ceph-mon[34495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:50:22.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:21 smithi089 ceph-mon[34495]: Cluster is now healthy 2024-05-08T05:50:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/2762561314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 msgr2=0x7fdbb4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/2762561314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb4102820 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009990 tx=0x7fdb9c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.657 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/2762561314 shutdown_connections 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/2762561314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4102d60 0x7fdbb410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/2762561314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb4108410 0x7fdbb41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/2762561314 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb4102820 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/2762561314 >> 172.21.15.3:0/2762561314 conn(0x7fdbb4079350 msgr2=0x7fdbb4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/2762561314 shutdown_connections 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.656+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/2762561314 wait complete. 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.657+0000 7fdbb8d8f700 1 Processor -- start 2024-05-08T05:50:22.658 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.657+0000 7fdbb8d8f700 1 -- start start 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4102d60 0x7fdbb419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb4108410 0x7fdbb41999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb41125d0 con 0x7fdbb41023a0 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4112450 con 0x7fdbb4102d60 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb4112750 con 0x7fdbb4108410 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44914/0 (socket says 172.21.15.3:44914) 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 -- 172.21.15.3:0/3133752243 learned_addr learned my addr 172.21.15.3:0/3133752243 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbabfff700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4102d60 0x7fdbb419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:22.660 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb2d9d700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb4108410 0x7fdbb41999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 -- 172.21.15.3:0/3133752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb4108410 msgr2=0x7fdbb41999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb4108410 0x7fdbb41999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 -- 172.21.15.3:0/3133752243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4102d60 msgr2=0x7fdbb419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4102d60 0x7fdbb419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 -- 172.21.15.3:0/3133752243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb419a120 con 0x7fdbb41023a0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb259c700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb419f800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdba400d980 tx=0x7fdba400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4017070 con 0x7fdbb41023a0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb419a410 con 0x7fdbb41023a0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba4005020 con 0x7fdbb41023a0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.658+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4009b50 con 0x7fdbb41023a0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.659+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb41a66d0 con 0x7fdbb41023a0 2024-05-08T05:50:22.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.659+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb406a190 con 0x7fdbb41023a0 2024-05-08T05:50:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.661+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fdba40051d0 con 0x7fdbb41023a0 2024-05-08T05:50:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.662+0000 7fdbab7fe700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdba00b2880 0x7fdba00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.662+0000 7fdbabfff700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdba00b2880 0x7fdba00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.662+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fdba40e27e0 con 0x7fdbb41023a0 2024-05-08T05:50:22.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.662+0000 7fdbabfff700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdba00b2880 0x7fdba00b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00b5c0 tx=0x7fdb9c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:22.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.665+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdba4020020 con 0x7fdbb41023a0 2024-05-08T05:50:22.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.925+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb41a68a0 con 0x7fdbb41023a0 2024-05-08T05:50:22.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.926+0000 7fdbab7fe700 1 -- 172.21.15.3:0/3133752243 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7fdba40aa2c0 con 0x7fdbb41023a0 2024-05-08T05:50:22.926 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:22.926 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:50:18.810108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdba00b2880 msgr2=0x7fdba00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdba00b2880 0x7fdba00b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c00b5c0 tx=0x7fdb9c005960 comp rx=0 tx=0).stop 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 msgr2=0x7fdbb419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb419f800 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdba400d980 tx=0x7fdba400dc90 comp rx=0 tx=0).stop 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 shutdown_connections 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbb4108410 0x7fdbb41999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdba00b2880 0x7fdba00b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdbb41023a0 0x7fdbb419f800 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 --2- 172.21.15.3:0/3133752243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdbb4102d60 0x7fdbb419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 >> 172.21.15.3:0/3133752243 conn(0x7fdbb4079350 msgr2=0x7fdbb4106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 shutdown_connections 2024-05-08T05:50:22.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:22.928+0000 7fdbb8d8f700 1 -- 172.21.15.3:0/3133752243 wait complete. 2024-05-08T05:50:22.930 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2024-05-08T05:50:23.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:22 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3133752243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:23.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:22 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3133752243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:23.313 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:22 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3133752243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:23.786 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:50:24.045 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:24.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:23 smithi003 ceph-mon[30137]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:24.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:23 smithi089 ceph-mon[34495]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:24.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:23 smithi104 ceph-mon[34559]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 -- 172.21.15.3:0/905698200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada8103540 msgr2=0x7fada81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 --2- 172.21.15.3:0/905698200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada8103540 0x7fada81039c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad9c0099c0 tx=0x7fad9c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 -- 172.21.15.3:0/905698200 shutdown_connections 2024-05-08T05:50:26.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 --2- 172.21.15.3:0/905698200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103f00 0x7fada81103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 --2- 172.21.15.3:0/905698200 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada81095e0 0x7fada81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 --2- 172.21.15.3:0/905698200 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada8103540 0x7fada81039c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 -- 172.21.15.3:0/905698200 >> 172.21.15.3:0/905698200 conn(0x7fada807a650 msgr2=0x7fada807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 -- 172.21.15.3:0/905698200 shutdown_connections 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.055+0000 7fadadd29700 1 -- 172.21.15.3:0/905698200 wait complete. 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.056+0000 7fadadd29700 1 Processor -- start 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.056+0000 7fadadd29700 1 -- start start 2024-05-08T05:50:26.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.056+0000 7fadadd29700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103540 0x7fada819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.056+0000 7fadadd29700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8103f00 0x7fada819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.056+0000 7fadadd29700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 0x7fada8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fadadd29700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fada8113740 con 0x7fada81095e0 2024-05-08T05:50:26.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fadadd29700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fada81135c0 con 0x7fada8103f00 2024-05-08T05:50:26.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fadadd29700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fada81138c0 con 0x7fada8103540 2024-05-08T05:50:26.061 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 0x7fada8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 0x7fada8199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41334/0 (socket says 172.21.15.3:41334) 2024-05-08T05:50:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 -- 172.21.15.3:0/599874090 learned_addr learned my addr 172.21.15.3:0/599874090 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada6ffd700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8103f00 0x7fada819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada77fe700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103540 0x7fada819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.062 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 -- 172.21.15.3:0/599874090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103540 msgr2=0x7fada819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103540 0x7fada819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 -- 172.21.15.3:0/599874090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8103f00 msgr2=0x7fada819fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8103f00 0x7fada819fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 -- 172.21.15.3:0/599874090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada819a1a0 con 0x7fada81095e0 2024-05-08T05:50:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada77fe700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103540 0x7fada819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:26.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada7fff700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 0x7fada8199a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fada0007ee0 tx=0x7fada000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada000dbd0 con 0x7fada81095e0 2024-05-08T05:50:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fada000dd70 con 0x7fada81095e0 2024-05-08T05:50:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada819a490 con 0x7fada81095e0 2024-05-08T05:50:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0005990 con 0x7fada81095e0 2024-05-08T05:50:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.057+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada81a6700 con 0x7fada81095e0 2024-05-08T05:50:26.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.058+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada810ba70 con 0x7fada81095e0 2024-05-08T05:50:26.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.061+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fada001a020 con 0x7fada81095e0 2024-05-08T05:50:26.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.062+0000 7fada4ff9700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fad900b2880 0x7fad900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:26.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.062+0000 7fada77fe700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fad900b2880 0x7fad900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.062+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7fada00e21e0 con 0x7fada81095e0 2024-05-08T05:50:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.062+0000 7fada77fe700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fad900b2880 0x7fad900b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fada8108fb0 tx=0x7fad98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:26.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.065+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fada0010070 con 0x7fada81095e0 2024-05-08T05:50:26.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.289+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fada819af20 con 0x7fada81095e0 2024-05-08T05:50:26.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.290+0000 7fada4ff9700 1 -- 172.21.15.3:0/599874090 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24045 (secure 0 0 0) 0x7fada00a9d60 con 0x7fada81095e0 2024-05-08T05:50:26.291 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:26.291 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":85,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:50:18.809958+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fad900b2880 msgr2=0x7fad900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fad900b2880 0x7fad900b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fada8108fb0 tx=0x7fad98008040 comp rx=0 tx=0).stop 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 msgr2=0x7fada8199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 0x7fada8199a40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fada0007ee0 tx=0x7fada000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 shutdown_connections 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fad900b2880 0x7fad900b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fada81095e0 0x7fada8199a40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fada8103540 0x7fada819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.294 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 --2- 172.21.15.3:0/599874090 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fada8103f00 0x7fada819fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 >> 172.21.15.3:0/599874090 conn(0x7fada807a650 msgr2=0x7fada8102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 shutdown_connections 2024-05-08T05:50:26.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:26.293+0000 7fadadd29700 1 -- 172.21.15.3:0/599874090 wait complete. 2024-05-08T05:50:26.305 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:26 smithi003 ceph-mon[30137]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:26.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:26 smithi104 ceph-mon[34559]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:26.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:26 smithi089 ceph-mon[34495]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:27.025 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:50:27.289 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:27.541 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:27 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/599874090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:27.613 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:27 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/599874090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:27.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:27 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/599874090' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:28.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:28 smithi003 ceph-mon[30137]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:28.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:28 smithi089 ceph-mon[34495]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:28.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:28 smithi104 ceph-mon[34559]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 -- 172.21.15.3:0/3701309228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c06c630 msgr2=0x7f887c110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:29.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 --2- 172.21.15.3:0/3701309228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c06c630 0x7f887c110360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f887400b0d0 tx=0x7f887400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 -- 172.21.15.3:0/3701309228 shutdown_connections 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 --2- 172.21.15.3:0/3701309228 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c06c630 0x7f887c110360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 --2- 172.21.15.3:0/3701309228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c109590 0x7f887c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 --2- 172.21.15.3:0/3701309228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c06bc70 0x7f887c06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 -- 172.21.15.3:0/3701309228 >> 172.21.15.3:0/3701309228 conn(0x7f887c079440 msgr2=0x7f887c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 -- 172.21.15.3:0/3701309228 shutdown_connections 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 -- 172.21.15.3:0/3701309228 wait complete. 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 Processor -- start 2024-05-08T05:50:29.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.043+0000 7f8881820700 1 -- start start 2024-05-08T05:50:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c06bc70 0x7f887c102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c06c630 0x7f887c102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 0x7f887c102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:29.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f887c113700 con 0x7f887c109590 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f887c113580 con 0x7f887c06bc70 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f887c113880 con 0x7f887c06c630 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 0x7f887c102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 0x7f887c102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41356/0 (socket says 172.21.15.3:41356) 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 -- 172.21.15.3:0/976527722 learned_addr learned my addr 172.21.15.3:0/976527722 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887affd700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c06bc70 0x7f887c102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887a7fc700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c06c630 0x7f887c102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 -- 172.21.15.3:0/976527722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c06c630 msgr2=0x7f887c102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c06c630 0x7f887c102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 -- 172.21.15.3:0/976527722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c06bc70 msgr2=0x7f887c102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c06bc70 0x7f887c102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 -- 172.21.15.3:0/976527722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c1a7fa0 con 0x7f887c109590 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887a7fc700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c06c630 0x7f887c102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:50:29.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f887b7fe700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 0x7f887c102e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8874009fd0 tx=0x7f8874007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887401f070 con 0x7f887c109590 2024-05-08T05:50:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887c1a8230 con 0x7f887c109590 2024-05-08T05:50:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.044+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8874003c00 con 0x7f887c109590 2024-05-08T05:50:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.045+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887401b8e0 con 0x7f887c109590 2024-05-08T05:50:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.045+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887c1a8680 con 0x7f887c109590 2024-05-08T05:50:29.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.045+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f887c10ba20 con 0x7f887c109590 2024-05-08T05:50:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.047+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f8874022070 con 0x7f887c109590 2024-05-08T05:50:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.048+0000 7f888081e700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f88640b2880 0x7f88640b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.048+0000 7f887affd700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f88640b2880 0x7f88640b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.048+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f887400f020 con 0x7f887c109590 2024-05-08T05:50:29.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.048+0000 7f887affd700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f88640b2880 0x7f88640b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f886c006fd0 tx=0x7f886c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:29.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.051+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8874029020 con 0x7f887c109590 2024-05-08T05:50:29.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.274+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f887c02e510 con 0x7f887c109590 2024-05-08T05:50:29.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.276+0000 7f888081e700 1 -- 172.21.15.3:0/976527722 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+24045 (secure 0 0 0) 0x7f88740ab760 con 0x7f887c109590 2024-05-08T05:50:29.277 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:29.277 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":85,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:50:18.809958+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:29.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f88640b2880 msgr2=0x7f88640b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:29.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f88640b2880 0x7f88640b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f886c006fd0 tx=0x7f886c008040 comp rx=0 tx=0).stop 2024-05-08T05:50:29.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 msgr2=0x7f887c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:29.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 0x7f887c102e90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8874009fd0 tx=0x7f8874007ad0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 shutdown_connections 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f88640b2880 0x7f88640b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f887c109590 0x7f887c102e90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f887c06bc70 0x7f887c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 --2- 172.21.15.3:0/976527722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f887c06c630 0x7f887c102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 >> 172.21.15.3:0/976527722 conn(0x7f887c079440 msgr2=0x7f887c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 shutdown_connections 2024-05-08T05:50:29.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:29.278+0000 7f8881820700 1 -- 172.21.15.3:0/976527722 wait complete. 2024-05-08T05:50:29.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:29 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/976527722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:29.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:29 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/976527722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:29.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:29 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/976527722' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:30.029 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:50:30.283 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:30.542 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:30 smithi003 ceph-mon[30137]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:30.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:30 smithi089 ceph-mon[34495]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:30.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:30 smithi104 ceph-mon[34559]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 -- 172.21.15.3:0/4284084113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 msgr2=0x7f9700108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 --2- 172.21.15.3:0/4284084113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f9700108140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f96e80099c0 tx=0x7f96e8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 -- 172.21.15.3:0/4284084113 shutdown_connections 2024-05-08T05:50:31.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 --2- 172.21.15.3:0/4284084113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9700108870 0x7f97001103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 --2- 172.21.15.3:0/4284084113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9700103780 0x7f9700103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 --2- 172.21.15.3:0/4284084113 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f9700108140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 -- 172.21.15.3:0/4284084113 >> 172.21.15.3:0/4284084113 conn(0x7f970007a610 msgr2=0x7f970007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 -- 172.21.15.3:0/4284084113 shutdown_connections 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.966+0000 7f9705de7700 1 -- 172.21.15.3:0/4284084113 wait complete. 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 Processor -- start 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 -- start start 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9700103780 0x7f970019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f970019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:31.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9700108870 0x7f97001999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9700113730 con 0x7f9700104130 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f97001135b0 con 0x7f9700103780 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f9705de7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f97001138b0 con 0x7f9700108870 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.967+0000 7f96f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f970019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f970019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41386/0 (socket says 172.21.15.3:41386) 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 -- 172.21.15.3:0/434499491 learned_addr learned my addr 172.21.15.3:0/434499491 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9700103780 0x7f970019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96fffff700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9700108870 0x7f97001999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 -- 172.21.15.3:0/434499491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9700108870 msgr2=0x7f97001999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9700108870 0x7f97001999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:31.971 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 -- 172.21.15.3:0/434499491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9700103780 msgr2=0x7f970019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9700103780 0x7f970019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 -- 172.21.15.3:0/434499491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970019a110 con 0x7f9700104130 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96f6ffd700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f970019fd30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b580 tx=0x7f96e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8027070 con 0x7f9700104130 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970019a3a0 con 0x7f9700104130 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e80040a0 con 0x7f9700104130 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e800f460 con 0x7f9700104130 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.968+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97001a66c0 con 0x7f9700104130 2024-05-08T05:50:31.972 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.969+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970006a190 con 0x7f9700104130 2024-05-08T05:50:31.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.971+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f96e802a070 con 0x7f9700104130 2024-05-08T05:50:31.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.971+0000 7f96fd7fa700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f96ec0b2880 0x7f96ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:31.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.971+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f96ec0b2880 0x7f96ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:31.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.972+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8802+0+0 (secure 0 0 0) 0x7f96e80e39c0 con 0x7f9700104130 2024-05-08T05:50:31.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.972+0000 7f96ff7fe700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f96ec0b2880 0x7f96ec0b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f970019aaf0 tx=0x7f96f000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:31.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:31.975+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f96e8031070 con 0x7f9700104130 2024-05-08T05:50:32.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.242+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f970010bab0 con 0x7f9700104130 2024-05-08T05:50:32.243 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.242+0000 7f96fd7fa700 1 -- 172.21.15.3:0/434499491 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11788 (secure 0 0 0) 0x7f96e8013070 con 0x7f9700104130 2024-05-08T05:50:32.243 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:32.243 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:50:18.810108+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.244+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f96ec0b2880 msgr2=0x7f96ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.244+0000 7f9705de7700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f96ec0b2880 0x7f96ec0b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f970019aaf0 tx=0x7f96f000d040 comp rx=0 tx=0).stop 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.244+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 msgr2=0x7f970019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.244+0000 7f9705de7700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f970019fd30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f96e800b580 tx=0x7f96e80047b0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 shutdown_connections 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9700108870 0x7f97001999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9700103780 0x7f970019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f96ec0b2880 0x7f96ec0b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 --2- 172.21.15.3:0/434499491 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9700104130 0x7f970019fd30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:32.245 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 >> 172.21.15.3:0/434499491 conn(0x7f970007a610 msgr2=0x7f9700101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 shutdown_connections 2024-05-08T05:50:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:32.245+0000 7f9705de7700 1 -- 172.21.15.3:0/434499491 wait complete. 2024-05-08T05:50:32.246 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 17 2024-05-08T05:50:32.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:32 smithi003 ceph-mon[30137]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:32.557 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:32 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/434499491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:32.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:32 smithi089 ceph-mon[34495]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:32.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:32 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/434499491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:32 smithi104 ceph-mon[34559]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:32.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:32 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/434499491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:50:32.921 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:50:33.172 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-08T05:50:33.544 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-08T05:50:33.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-08T05:50:33.545 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:33 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T05:50:33.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-08T05:50:33.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:33 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-05-08T05:50:33.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-05-08T05:50:33.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-05-08T05:50:33.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-08T05:50:33.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:33 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-08T05:50:34.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-08T05:50:34.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-08T05:50:34.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-08T05:50:34.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T05:50:34.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:34 smithi003 ceph-mon[30137]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-08T05:50:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T05:50:34.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:34 smithi089 ceph-mon[34495]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-08T05:50:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T05:50:34.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:34 smithi104 ceph-mon[34559]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/1078812521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 msgr2=0x7f9df8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/1078812521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df8110330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9df000b0a0 tx=0x7f9df000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/1078812521 shutdown_connections 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/1078812521 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df8110330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/1078812521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 0x7f9df81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/1078812521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9df806b180 0x7f9df806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/1078812521 >> 172.21.15.3:0/1078812521 conn(0x7f9df8079350 msgr2=0x7f9df8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:34.910 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/1078812521 shutdown_connections 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.907+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/1078812521 wait complete. 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.908+0000 7f9dfeaa4700 1 Processor -- start 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.908+0000 7f9dfeaa4700 1 -- start start 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfeaa4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9df806b180 0x7f9df81a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfeaa4700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df81a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfeaa4700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 0x7f9df810f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfeaa4700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9df8113640 con 0x7f9df806bb40 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfeaa4700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81134c0 con 0x7f9df8109510 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfeaa4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9df81137c0 con 0x7f9df806b180 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df81a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df81a5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51130/0 (socket says 172.21.15.3:51130) 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 -- 172.21.15.3:0/491781913 learned_addr learned my addr 172.21.15.3:0/491781913 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfd041700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 0x7f9df810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:34.911 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfc840700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9df806b180 0x7f9df81a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 -- 172.21.15.3:0/491781913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9df806b180 msgr2=0x7f9df81a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9df806b180 0x7f9df81a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 -- 172.21.15.3:0/491781913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 msgr2=0x7f9df810f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 0x7f9df810f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 -- 172.21.15.3:0/491781913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df810fd20 con 0x7f9df806bb40 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9dfd041700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 0x7f9df810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.909+0000 7f9df7fff700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df81a5290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c970 tx=0x7f9de800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.910+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de80078d0 con 0x7f9df806bb40 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.910+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df8110010 con 0x7f9df806bb40 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.910+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de8018480 con 0x7f9df806bb40 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.910+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de800f6b0 con 0x7f9df806bb40 2024-05-08T05:50:34.912 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.910+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df81acc80 con 0x7f9df806bb40 2024-05-08T05:50:34.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.911+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df806a190 con 0x7f9df806bb40 2024-05-08T05:50:34.913 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.912+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f9de8018930 con 0x7f9df806bb40 2024-05-08T05:50:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.913+0000 7f9df5ffb700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9de40b2880 0x7f9de40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.913+0000 7f9dfc840700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9de40b2880 0x7f9de40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.913+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9de80e26d0 con 0x7f9df806bb40 2024-05-08T05:50:34.914 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.913+0000 7f9dfc840700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9de40b2880 0x7f9de40b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9df80798c0 tx=0x7f9de000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:34.918 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:34.918+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9de80aa0c0 con 0x7f9df806bb40 2024-05-08T05:50:35.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.132+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9df81aceb0 con 0x7f9df806bb40 2024-05-08T05:50:35.134 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.133+0000 7f9df5ffb700 1 -- 172.21.15.3:0/491781913 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24572 (secure 0 0 0) 0x7f9de80a99c0 con 0x7f9df806bb40 2024-05-08T05:50:35.135 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:35.135 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:50:34.293465+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:35.136 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9de40b2880 msgr2=0x7f9de40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9de40b2880 0x7f9de40b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9df80798c0 tx=0x7f9de000b410 comp rx=0 tx=0).stop 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 msgr2=0x7f9df81a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df81a5290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c970 tx=0x7f9de800cc80 comp rx=0 tx=0).stop 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 shutdown_connections 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9df8109510 0x7f9df810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9df806b180 0x7f9df81a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9de40b2880 0x7f9de40b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 --2- 172.21.15.3:0/491781913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9df806bb40 0x7f9df81a5290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.135+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 >> 172.21.15.3:0/491781913 conn(0x7f9df8079350 msgr2=0x7f9df8100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.136+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 shutdown_connections 2024-05-08T05:50:35.137 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:35.136+0000 7f9dfeaa4700 1 -- 172.21.15.3:0/491781913 wait complete. 2024-05-08T05:50:35.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:35 smithi003 ceph-mon[30137]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T05:50:35.447 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:35 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/491781913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:35.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:50:01.809365+0000', 'modified': '2024-05-08T05:50:18.810108+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.104:6839/3320987620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3320987620}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3320987620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24493, 'qdb_cluster': [24493]} 2024-05-08T05:50:35.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:50:35.762 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:50:35.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:35 smithi089 ceph-mon[34495]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T05:50:35.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:35 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/491781913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:35 smithi104 ceph-mon[34559]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T05:50:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:35 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/491781913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:36.013 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:36.391 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:36 smithi003 ceph-mon[30137]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:36.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:36 smithi089 ceph-mon[34495]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:36.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:36 smithi104 ceph-mon[34559]: pgmap v318: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:37.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 -- 172.21.15.3:0/3170601409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4103750 msgr2=0x7f01c4103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/3170601409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4103750 0x7f01c4103b30 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f01b40099c0 tx=0x7f01b4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 -- 172.21.15.3:0/3170601409 shutdown_connections 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/3170601409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01c4108850 0x7f01c41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/3170601409 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4103750 0x7f01c4103b30 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/3170601409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01c4104100 0x7f01c4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 -- 172.21.15.3:0/3170601409 >> 172.21.15.3:0/3170601409 conn(0x7f01c407a600 msgr2=0x7f01c407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 -- 172.21.15.3:0/3170601409 shutdown_connections 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 -- 172.21.15.3:0/3170601409 wait complete. 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 Processor -- start 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.727+0000 7f01ca1cc700 1 -- start start 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01ca1cc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01c4103750 0x7f01c41a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01ca1cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 0x7f01c41a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01ca1cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01c4108850 0x7f01c419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01ca1cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f01c4113620 con 0x7f01c4104100 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01ca1cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f01c41134a0 con 0x7f01c4108850 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01ca1cc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f01c41137a0 con 0x7f01c4103750 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 0x7f01c41a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 0x7f01c41a1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51156/0 (socket says 172.21.15.3:51156) 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 -- 172.21.15.3:0/4150345788 learned_addr learned my addr 172.21.15.3:0/4150345788 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01c4103750 0x7f01c41a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01c3fff700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01c4108850 0x7f01c419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 -- 172.21.15.3:0/4150345788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01c4103750 msgr2=0x7f01c41a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01c4103750 0x7f01c41a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 -- 172.21.15.3:0/4150345788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01c4108850 msgr2=0x7f01c419bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01c4108850 0x7f01c419bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 -- 172.21.15.3:0/4150345788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c419c280 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01bbfff700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 0x7f01c41a1e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c920 tx=0x7f01ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.728+0000 7f01c17fa700 1 -- 172.21.15.3:0/4150345788 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac004a30 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.729+0000 7f01c17fa700 1 -- 172.21.15.3:0/4150345788 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ac0054d0 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.729+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c419c570 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.729+0000 7f01c17fa700 1 -- 172.21.15.3:0/4150345788 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac007c70 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.729+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c41a87e0 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.731+0000 7f01c17fa700 1 -- 172.21.15.3:0/4150345788 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f01ac004c60 con 0x7f01c4104100 2024-05-08T05:50:37.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01c17fa700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f01b00b2880 0x7f01b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f01b00b2880 0x7f01b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01c17fa700 1 -- 172.21.15.3:0/4150345788 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f01ac0e2440 con 0x7f01c4104100 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 Processor -- start 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 -- start start 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f01b00b2880 0x7f01b00b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009990 tx=0x7f01b400b870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84687540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687a80 0x562b846817c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84681d00 0x562b84682140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b845d38f0 con 0x562b84687160 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b84538d50 con 0x562b84687a80 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.732+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b845cf8b0 con 0x562b84681d00 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84687540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84687540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51160/0 (socket says 172.21.15.3:51160) 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 -- 172.21.15.3:0/2592167313 learned_addr learned my addr 172.21.15.3:0/2592167313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01bbfff700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687a80 0x562b846817c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c3fff700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84681d00 0x562b84682140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 -- 172.21.15.3:0/2592167313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84681d00 msgr2=0x562b84682140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84681d00 0x562b84682140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 -- 172.21.15.3:0/2592167313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687a80 msgr2=0x562b846817c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687a80 0x562b846817c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 -- 172.21.15.3:0/2592167313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b84520880 con 0x562b84687160 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c3fff700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84681d00 0x562b84682140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01bbfff700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687a80 0x562b846817c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01c37fe700 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84687540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01b4015000 tx=0x7f01b401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9109cc3670574b92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01b9ffb700 1 -- 172.21.15.3:0/2592167313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035070 con 0x562b84687160 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01b9ffb700 1 -- 172.21.15.3:0/2592167313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b4025a40 con 0x562b84687160 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.733+0000 7f01b9ffb700 1 -- 172.21.15.3:0/2592167313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b400fab0 con 0x562b84687160 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2592167313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 msgr2=0x562b84687540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84687540 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01b4015000 tx=0x7f01b401f000 comp rx=0 tx=0).stop 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2592167313 shutdown_connections 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84681d00 0x562b84682140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84687540 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2592167313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687a80 0x562b846817c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2592167313 >> 172.21.15.3:0/2592167313 conn(0x562b845adb20 msgr2=0x562b8467c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2592167313 shutdown_connections 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.734+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2592167313 wait complete. 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 -- start start 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 Processor -- start 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b8468d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687160 0x562b8468aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 0x562b8468afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b8468e3a0 con 0x562b84687a80 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b8458ce80 con 0x562b84681d00 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b8468e520 con 0x562b84687160 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01d4a5fb80 10 client.0 _ll_register_callbacks cb 0x562b84567560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:37.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01c3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 0x562b8468afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01c3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 0x562b8468afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51162/0 (socket says 172.21.15.3:51162) 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01c3fff700 1 -- 172.21.15.3:0/406232201 learned_addr learned my addr 172.21.15.3:0/406232201 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01bbfff700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687160 0x562b8468aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.735+0000 7f01c37fe700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b8468d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01c3fff700 1 -- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687160 msgr2=0x562b8468aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01c3fff700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687160 0x562b8468aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01c3fff700 1 -- 172.21.15.3:0/406232201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 msgr2=0x562b8468d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01c3fff700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b8468d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01c3fff700 1 -- 172.21.15.3:0/406232201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8458a710 con 0x562b84687a80 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01c3fff700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 0x562b8468afc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01bc011cb0 tx=0x7f01bc00f1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37a709d29083d62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01a2ffd700 1 -- 172.21.15.3:0/406232201 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc01b020 con 0x562b84687a80 2024-05-08T05:50:37.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01a2ffd700 1 -- 172.21.15.3:0/406232201 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f01bc00fa60 con 0x562b84687a80 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b4025a40 con 0x562b84687a80 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01a2ffd700 1 -- 172.21.15.3:0/406232201 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f01bc026050 con 0x562b84687a80 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01d4a5fb80 10 client.14679 fetch_fsmap learned FSMap version 17 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562b84698cb0 con 0x562b84687a80 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01a2ffd700 1 -- 172.21.15.3:0/406232201 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b4025a40 con 0x562b84687a80 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.736+0000 7f01a2ffd700 1 -- 172.21.15.3:0/406232201 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f01bc015430 con 0x562b84687a80 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01d4a5fb80 10 client.14679 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01d4a5fb80 10 client.14679 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01d4a5fb80 20 client.14679 populate_metadata read hostname 'smithi003' 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8469c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01d4a5fb80 4 client.14679 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b845e12d0 con 0x562b8469b330 2024-05-08T05:50:37.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01c37fe700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8469c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01a2ffd700 10 client.14679 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.737+0000 7f01c37fe700 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8469c7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3ec7973e4b5ab1 server_cookie=fdb87d673f840ccf in_seq=0 out_seq=0 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01a2ffd700 1 -- 172.21.15.3:0/406232201 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562b845e12d0 con 0x562b8469b330 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01a2ffd700 10 client.14679 handle_command_reply: tid=0 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 client.14679 shutdown 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 msgr2=0x562b8468afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 0x562b8468afc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f01bc011cb0 tx=0x7f01bc00f1d0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 shutdown_connections 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687a80 0x562b8468afc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8469c7a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01a2ffd700 0 client.14679 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b8468d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/406232201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687160 0x562b8468aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 >> 172.21.15.3:0/406232201 conn(0x562b845adb20 msgr2=0x562b84676ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:37.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.738+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 shutdown_connections 2024-05-08T05:50:37.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.739+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/406232201 wait complete. 2024-05-08T05:50:37.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.739+0000 7f01d4a5fb80 20 client.14679 trim_cache size 0 max 16384 2024-05-08T05:50:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.759+0000 7f01d4a5fb80 1 Processor -- start 2024-05-08T05:50:37.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.759+0000 7f01d4a5fb80 1 -- start start 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 0x562b8468aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687160 0x562b84676bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 0x562b84687e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b8469aae0 con 0x562b84681d00 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b8469b1b0 con 0x562b84687160 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b8469b880 con 0x562b84687a80 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 0x562b8468aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 0x562b8468aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51168/0 (socket says 172.21.15.3:51168) 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 -- 172.21.15.3:0/2935604086 learned_addr learned my addr 172.21.15.3:0/2935604086 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 -- 172.21.15.3:0/2935604086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 msgr2=0x562b84687e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 0x562b84687e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 -- 172.21.15.3:0/2935604086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687160 msgr2=0x562b84676bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687160 0x562b84676bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 -- 172.21.15.3:0/2935604086 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8468b960 con 0x562b84681d00 2024-05-08T05:50:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01c37fe700 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 0x562b8468aff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f01b400bbf0 tx=0x7f01b400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d790fad7836f3b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01b97fa700 1 -- 172.21.15.3:0/2935604086 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4035040 con 0x562b84681d00 2024-05-08T05:50:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01b97fa700 1 -- 172.21.15.3:0/2935604086 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b40257b0 con 0x562b84681d00 2024-05-08T05:50:37.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.760+0000 7f01b97fa700 1 -- 172.21.15.3:0/2935604086 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b403d3c0 con 0x562b84681d00 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2935604086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 msgr2=0x562b8468aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 0x562b8468aff0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f01b400bbf0 tx=0x7f01b400b5c0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2935604086 shutdown_connections 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 0x562b84687e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84681d00 0x562b8468aff0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/2935604086 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84687160 0x562b84676bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2935604086 >> 172.21.15.3:0/2935604086 conn(0x562b845adb20 msgr2=0x562b84689370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2935604086 shutdown_connections 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/2935604086 wait complete. 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.761+0000 7f01d4a5fb80 1 -- start start 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 Processor -- start 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b846a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.766 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84675ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 0x562b846762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x562b8469c620 con 0x562b84687160 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562b8469bf50 con 0x562b84681d00 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b8469ccf0 con 0x562b84687a80 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01d4a5fb80 10 client.0 _ll_register_callbacks cb 0x562b8467e3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84675ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84675ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51174/0 (socket says 172.21.15.3:51174) 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 -- 172.21.15.3:0/4235417713 learned_addr learned my addr 172.21.15.3:0/4235417713 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b846a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 -- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 msgr2=0x562b846762b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 0x562b846762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 -- 172.21.15.3:0/4235417713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 msgr2=0x562b846a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b846a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01bbfff700 1 -- 172.21.15.3:0/4235417713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b84676ac0 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.762+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b846a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01bbfff700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84675ca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f01ac004fb0 tx=0x7f01ac0a92c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff16167c09e80d48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01a2ffd700 1 -- 172.21.15.3:0/4235417713 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ac01d040 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01a2ffd700 1 -- 172.21.15.3:0/4235417713 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f01ac0a5990 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b40257b0 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01a2ffd700 1 -- 172.21.15.3:0/4235417713 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f01ac06a050 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 10 client.14685 fetch_fsmap learned FSMap version 17 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562b847068b0 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01a2ffd700 1 -- 172.21.15.3:0/4235417713 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b40257b0 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01a2ffd700 1 -- 172.21.15.3:0/4235417713 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f01ac0a9960 con 0x562b84687160 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 10 client.14685 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 10 client.14685 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 20 client.14685 populate_metadata read hostname 'smithi003' 2024-05-08T05:50:37.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8470a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 4 client.14685 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.763+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "status"}) -- 0x562b845db370 con 0x562b8469b330 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8470a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01c37fe700 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8470a230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae41e7429866a0a8 server_cookie=ce48ffd2fc065014 in_seq=0 out_seq=0 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01a2ffd700 10 client.14685 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01a2ffd700 1 -- 172.21.15.3:0/4235417713 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b845db370 con 0x562b8469b330 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01a2ffd700 10 client.14685 handle_command_reply: tid=0 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 client.14685 shutdown 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 msgr2=0x562b84675ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84675ca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f01ac004fb0 tx=0x7f01ac0a92c0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 shutdown_connections 2024-05-08T05:50:37.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b84687a80 0x562b846762b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x562b8469b330 0x562b8470a230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562b84681d00 0x562b846a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 --2- 172.21.15.3:0/4235417713 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x562b84687160 0x562b84675ca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 >> 172.21.15.3:0/4235417713 conn(0x562b845adb20 msgr2=0x562b84687e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.764+0000 7f01a2ffd700 0 client.14685 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 shutdown_connections 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01d4a5fb80 1 -- 172.21.15.3:0/4235417713 wait complete. 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01d4a5fb80 20 client.14685 trim_cache size 0 max 16384 2024-05-08T05:50:37.770 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 35.125441490999997, 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 17, 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 87, 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 85, 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 64.277960581000002 2024-05-08T05:50:37.771 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f01b00b2880 msgr2=0x7f01b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f01b00b2880 0x7f01b00b4d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009990 tx=0x7f01b400b870 comp rx=0 tx=0).stop 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 msgr2=0x7f01c41a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 0x7f01c41a1e50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c920 tx=0x7f01ac00cc30 comp rx=0 tx=0).stop 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 shutdown_connections 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f01c4108850 0x7f01c419bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01c4103750 0x7f01c41a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f01b00b2880 0x7f01b00b4d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 --2- 172.21.15.3:0/4150345788 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f01c4104100 0x7f01c41a1e50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:37.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 >> 172.21.15.3:0/4150345788 conn(0x7f01c407a600 msgr2=0x7f01c4101910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:37.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 shutdown_connections 2024-05-08T05:50:37.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:37.765+0000 7f01ca1cc700 1 -- 172.21.15.3:0/4150345788 wait complete. 2024-05-08T05:50:38.604 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:38 smithi003 ceph-mon[30137]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:38.639 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:50:38.639 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:38.639 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T05:50:38.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:38 smithi089 ceph-mon[34495]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:38.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:38 smithi104 ceph-mon[34559]: pgmap v319: 129 pgs: 2 peering, 127 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:38.898 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:40.570 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:40 smithi003 ceph-mon[30137]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 -- 172.21.15.3:0/285403934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806c590 msgr2=0x7fd638110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 --2- 172.21.15.3:0/285403934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806c590 0x7fd638110330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd63000b0a0 tx=0x7fd63000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 -- 172.21.15.3:0/285403934 shutdown_connections 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 --2- 172.21.15.3:0/285403934 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806c590 0x7fd638110330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 --2- 172.21.15.3:0/285403934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd638109510 0x7fd6381098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 --2- 172.21.15.3:0/285403934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd63806bbd0 0x7fd63806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 -- 172.21.15.3:0/285403934 >> 172.21.15.3:0/285403934 conn(0x7fd638079350 msgr2=0x7fd638079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 -- 172.21.15.3:0/285403934 shutdown_connections 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.570+0000 7fd63e74f700 1 -- 172.21.15.3:0/285403934 wait complete. 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 Processor -- start 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 -- start start 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 0x7fd63819bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:40.573 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63806c590 0x7fd63819c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd638109510 0x7fd6381a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd6381136c0 con 0x7fd63806bbd0 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd638113540 con 0x7fd63806c590 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63e74f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd638113840 con 0x7fd638109510 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd637fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 0x7fd63819bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd637fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 0x7fd63819bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51184/0 (socket says 172.21.15.3:51184) 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd637fff700 1 -- 172.21.15.3:0/2721165980 learned_addr learned my addr 172.21.15.3:0/2721165980 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd63ccec700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd638109510 0x7fd6381a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.571+0000 7fd6377fe700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63806c590 0x7fd63819c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd637fff700 1 -- 172.21.15.3:0/2721165980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd638109510 msgr2=0x7fd6381a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd637fff700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd638109510 0x7fd6381a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd637fff700 1 -- 172.21.15.3:0/2721165980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63806c590 msgr2=0x7fd63819c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd637fff700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63806c590 0x7fd63819c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd637fff700 1 -- 172.21.15.3:0/2721165980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6381a0d90 con 0x7fd63806bbd0 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd637fff700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 0x7fd63819bd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b530 tx=0x7fd62800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628015020 con 0x7fd63806bbd0 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd628011ce0 con 0x7fd63806bbd0 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6381a1080 con 0x7fd63806bbd0 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628010e40 con 0x7fd63806bbd0 2024-05-08T05:50:40.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.572+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6381a8900 con 0x7fd63806bbd0 2024-05-08T05:50:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.573+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63806a190 con 0x7fd63806bbd0 2024-05-08T05:50:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.575+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd628018020 con 0x7fd63806bbd0 2024-05-08T05:50:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.575+0000 7fd6357fa700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd6200b2880 0x7fd6200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.576+0000 7fd6377fe700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd6200b2880 0x7fd6200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.576+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd6280e1dc0 con 0x7fd63806bbd0 2024-05-08T05:50:40.576 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.576+0000 7fd6377fe700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd6200b2880 0x7fd6200b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:40.580 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.580+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd62801f020 con 0x7fd63806bbd0 2024-05-08T05:50:40.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:40 smithi089 ceph-mon[34495]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:40 smithi104 ceph-mon[34559]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:40.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.842+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd6381a8b90 con 0x7fd63806bbd0 2024-05-08T05:50:40.843 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.843+0000 7fd6357fa700 1 -- 172.21.15.3:0/2721165980 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fd6280a9780 con 0x7fd63806bbd0 2024-05-08T05:50:40.843 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:50:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd6200b2880 msgr2=0x7fd6200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:40.845 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd6200b2880 0x7fd6200b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd62c005950 tx=0x7fd62c0058e0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 msgr2=0x7fd63819bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 0x7fd63819bd80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b530 tx=0x7fd62800b840 comp rx=0 tx=0).stop 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 shutdown_connections 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd638109510 0x7fd6381a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd6200b2880 0x7fd6200b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd63806bbd0 0x7fd63819bd80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 --2- 172.21.15.3:0/2721165980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd63806c590 0x7fd63819c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 >> 172.21.15.3:0/2721165980 conn(0x7fd638079350 msgr2=0x7fd63810e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 shutdown_connections 2024-05-08T05:50:40.846 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:40.845+0000 7fd63e74f700 1 -- 172.21.15.3:0/2721165980 wait complete. 2024-05-08T05:50:41.501 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:41 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2721165980' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:50:41.529 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:50:41.529 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:50:41.529 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-05-08T05:50:41.529 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T05:50:41.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:41.530 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:50:41.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:50:41.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:50:41.530 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T05:50:41.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:41.530 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-05-08T05:50:41.547 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:50:41.547 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-05-08T05:50:41.548 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 42148sec preferred_lft 42148sec 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-05-08T05:50:41.549 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-05-08T05:50:41.550 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:50:41.550 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:50:41.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:50:41.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:41.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:41.743 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:50:41.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:41 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2721165980' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:50:41.770 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-05-08T05:50:41.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:41 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2721165980' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:50:41.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:41.772 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-05-08T05:50:41.846 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-05-08T05:50:41.846 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:50:41.846 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:50:41.846 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:50:41.846 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:50:41.847 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:50:41.847 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:50:41.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:50:41.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:50:41.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:50:41.847 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:50:41.936 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:50:42.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:42.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:42.162 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-08T05:50:42.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:42.179 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-05-08T05:50:42.236 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:50:42.236 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:50:42.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:50:42.237 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T05:50:42.237 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:50:42.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:50:42.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:42.361 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T05:50:42.362 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:50:42.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:50:42.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:42.503 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:50:42.503 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:50:42.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-05-08T05:50:42.503 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:50:42.503 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:50:42.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:50:42.575 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:42 smithi003 ceph-mon[30137]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:42.578 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:42.584 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T05:50:42.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:50:42.584 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:42.600 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T05:50:42.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:50:42.601 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:42.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-05-08T05:50:42.723 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-05-08T05:50:42.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:42 smithi089 ceph-mon[34495]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:42.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:42 smithi104 ceph-mon[34559]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-05-08T05:50:42.781 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-05-08T05:50:42.782 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/57ee78c12b223261b9c82c4ee8eede80dc882cb7cb669ce87249f012366e4608/merged 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d11e0200615e4bf9cd0647f78679d037705ac7825f7ba0a12ab7a28a5babfb8d/merged 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-05-08T05:50:42.783 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/428e12a219ca223073d3de86e1bff0b4e822d70f54dcbf79bd1f3873a21e452c/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/1e51fb652e6a9bde266293d2f6123cbcd81fac037890469cbaae0bafe668b5ea/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/566c54d2a453dde747a2b1685a971f01a3c9666352c75351202f5659ec8256bb/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/42f0b37bf7d756d21af522534fbecc2663a614f5952760bc97436760e12a60b7/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/6a94c2d7b24c50c84e9f925414cc84d402dc5b95373b791c85863bbdae5f0c43/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/8a7831d33e4791524adcabcfc70eb064bda792fb972420abb4afb6f255b15642/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/24144dbabd23c9eeb7721c419cd5923a5416ce6e06dfd930cd15d71e18ad094e/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/fa8fd93e6e43de069e25add78552362673b857890db7a1a63f323cd6026216c6/merged 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:50:42.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:42.785 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-05-08T05:50:42.836 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T05:50:42.836 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T05:50:42.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo modprobe fuse 2024-05-08T05:50:42.905 DEBUG:teuthology.orchestra.run.smithi003:> cat /proc/self/mounts | awk '{print $2}' 2024-05-08T05:50:42.963 INFO:teuthology.orchestra.run.smithi003.stdout:/proc 2024-05-08T05:50:42.963 INFO:teuthology.orchestra.run.smithi003.stdout:/sys 2024-05-08T05:50:42.963 INFO:teuthology.orchestra.run.smithi003.stdout:/dev 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/security 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/shm 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/pts 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/run 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/cgroup 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/pstore 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/bpf 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/ 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/selinux 2024-05-08T05:50:42.964 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/hugepages 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/dev/mqueue 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/debug 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/tracing 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/fs/fuse/connections 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/sys/kernel/config 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-sysctl.service 2024-05-08T05:50:42.965 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/nfs/rpc_pipefs 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/1000 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/ceph 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/57ee78c12b223261b9c82c4ee8eede80dc882cb7cb669ce87249f012366e4608/merged 2024-05-08T05:50:42.966 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/d11e0200615e4bf9cd0647f78679d037705ac7825f7ba0a12ab7a28a5babfb8d/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/run/user/0 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/428e12a219ca223073d3de86e1bff0b4e822d70f54dcbf79bd1f3873a21e452c/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/1e51fb652e6a9bde266293d2f6123cbcd81fac037890469cbaae0bafe668b5ea/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/566c54d2a453dde747a2b1685a971f01a3c9666352c75351202f5659ec8256bb/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/42f0b37bf7d756d21af522534fbecc2663a614f5952760bc97436760e12a60b7/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/6a94c2d7b24c50c84e9f925414cc84d402dc5b95373b791c85863bbdae5f0c43/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/8a7831d33e4791524adcabcfc70eb064bda792fb972420abb4afb6f255b15642/merged 2024-05-08T05:50:42.967 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/24144dbabd23c9eeb7721c419cd5923a5416ce6e06dfd930cd15d71e18ad094e/merged 2024-05-08T05:50:42.968 INFO:teuthology.orchestra.run.smithi003.stdout:/var/lib/containers/storage/overlay/fa8fd93e6e43de069e25add78552362673b857890db7a1a63f323cd6026216c6/merged 2024-05-08T05:50:42.968 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns 2024-05-08T05:50:42.968 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:50:42.968 INFO:teuthology.orchestra.run.smithi003.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:50:42.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:42.968 DEBUG:teuthology.orchestra.run.smithi003:> ls /sys/fs/fuse/connections 2024-05-08T05:50:42.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:2024-05-08T05:50:42.988+0000 7f3728561480 -1 init, newargv = 0x5629ba3e4340 newargc=15 2024-05-08T05:50:42.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[76652]: starting ceph client 2024-05-08T05:50:42.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[76652]: starting fuse 2024-05-08T05:50:43.004 INFO:teuthology.orchestra.run.smithi003.stdout:114 2024-05-08T05:50:43.004 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [114] 2024-05-08T05:50:43.004 DEBUG:teuthology.orchestra.run.smithi003:> sudo stdin-killer -- python3 -c ' 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> import glob 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> import re 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> import os 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> import subprocess 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> def _find_admin_socket(client_name): 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> files = glob.glob(asok_path) 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> # Given a non-glob path, it better be there 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> if "*" not in asok_path: 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> assert(len(files) == 1) 2024-05-08T05:50:43.005 DEBUG:teuthology.orchestra.run.smithi003:> return files[0] 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> for f in files: 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> contents = proc_f.read() 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> if mountpoint in contents: 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> return f 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> print(_find_admin_socket("client.admin")) 2024-05-08T05:50:43.006 DEBUG:teuthology.orchestra.run.smithi003:> ' 2024-05-08T05:50:43.127 INFO:teuthology.orchestra.run.smithi003.stdout:/var/run/ceph/ceph-client.admin.76652.asok 2024-05-08T05:50:43.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:43.138 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76652.asok 2024-05-08T05:50:43.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:43.138 DEBUG:teuthology.orchestra.run.smithi003:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76652.asok status 2024-05-08T05:50:43.277 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "metadata": { 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_sha1": "bee299f4ac344e4c45e550d7a80abcadd7bc3d5a", 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "entity_id": "admin", 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": "smithi003", 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "pid": "76652", 2024-05-08T05:50:43.278 INFO:teuthology.orchestra.run.smithi003.stdout: "root": "/" 2024-05-08T05:50:43.279 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:50:43.279 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_count": 0, 2024-05-08T05:50:43.279 INFO:teuthology.orchestra.run.smithi003.stdout: "dentry_pinned_count": 0, 2024-05-08T05:50:43.279 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 14697, 2024-05-08T05:50:43.279 INFO:teuthology.orchestra.run.smithi003.stdout: "inst": { 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": { 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "client", 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "num": 14697 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1088811858 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:50:43.280 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": { 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "192.168.0.1:0", 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1088811858 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "inst_str": "client.14697 192.168.0.1:0/1088811858", 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "addr_str": "192.168.0.1:0/1088811858", 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "inode_count": 1, 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_epoch": 17, 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch": 87, 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_epoch_barrier": 0, 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "blocklisted": false, 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs" 2024-05-08T05:50:43.281 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:50:43.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:43.286 DEBUG:teuthology.orchestra.run.smithi003:> stat --file-system '--printf=%T 2024-05-08T05:50:43.286 DEBUG:teuthology.orchestra.run.smithi003:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:43.303 INFO:teuthology.orchestra.run.smithi003.stdout:fuseblk 2024-05-08T05:50:43.303 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:43.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:43.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:43.372 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T05:50:43.465 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:50:43.491 INFO:teuthology.orchestra.run.smithi003.stdout:# file: . 2024-05-08T05:50:43.491 INFO:teuthology.orchestra.run.smithi003.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T05:50:43.491 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:43.491 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:50:43.497 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:50:43.497 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:43.515 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T05:50:43.806 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:44.636 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:44 smithi003 ceph-mon[30137]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:44.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:44 smithi089 ceph-mon[34495]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:44.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:44 smithi104 ceph-mon[34559]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 -- 172.21.15.3:0/3432893913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724102fd0 msgr2=0x7fd724107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 --2- 172.21.15.3:0/3432893913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724102fd0 0x7fd724107020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd7180099c0 tx=0x7fd718009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 -- 172.21.15.3:0/3432893913 shutdown_connections 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 --2- 172.21.15.3:0/3432893913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd724107700 0x7fd72410f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 --2- 172.21.15.3:0/3432893913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102620 0x7fd724102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 --2- 172.21.15.3:0/3432893913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724102fd0 0x7fd724107020 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 -- 172.21.15.3:0/3432893913 >> 172.21.15.3:0/3432893913 conn(0x7fd724079440 msgr2=0x7fd724079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 -- 172.21.15.3:0/3432893913 shutdown_connections 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.514+0000 7fd729f4e700 1 -- 172.21.15.3:0/3432893913 wait complete. 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 Processor -- start 2024-05-08T05:50:45.517 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 -- start start 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd724102620 0x7fd724198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102fd0 0x7fd724198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 0x7fd7241992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd724112590 con 0x7fd724107700 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd724112410 con 0x7fd724102fd0 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd729f4e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd724112710 con 0x7fd724102620 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd723fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 0x7fd7241992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd723fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 0x7fd7241992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36796/0 (socket says 172.21.15.3:36796) 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd723fff700 1 -- 172.21.15.3:0/1990539960 learned_addr learned my addr 172.21.15.3:0/1990539960 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd7237fe700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd724102620 0x7fd724198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.515+0000 7fd722ffd700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102fd0 0x7fd724198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd723fff700 1 -- 172.21.15.3:0/1990539960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd724102620 msgr2=0x7fd724198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd723fff700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd724102620 0x7fd724198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd723fff700 1 -- 172.21.15.3:0/1990539960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102fd0 msgr2=0x7fd724198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd723fff700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102fd0 0x7fd724198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd723fff700 1 -- 172.21.15.3:0/1990539960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7241a8030 con 0x7fd724107700 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd722ffd700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102fd0 0x7fd724198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd723fff700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 0x7fd7241992a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd71c007ee0 tx=0x7fd71c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00dbd0 con 0x7fd724107700 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd71c00dd70 con 0x7fd724107700 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7241a8320 con 0x7fd724107700 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7241a87e0 con 0x7fd724107700 2024-05-08T05:50:45.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.516+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c005990 con 0x7fd724107700 2024-05-08T05:50:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.517+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72406a190 con 0x7fd724107700 2024-05-08T05:50:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.520+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fd71c01a020 con 0x7fd724107700 2024-05-08T05:50:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.520+0000 7fd720ff9700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd70c0b2880 0x7fd70c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.520+0000 7fd7237fe700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd70c0b2880 0x7fd70c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:45.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.520+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd71c0e2340 con 0x7fd724107700 2024-05-08T05:50:45.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.521+0000 7fd7237fe700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd70c0b2880 0x7fd70c0b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd714005950 tx=0x7fd71400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:45.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.524+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd71c010070 con 0x7fd724107700 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.758+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd72402e510 con 0x7fd724107700 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.759+0000 7fd720ff9700 1 -- 172.21.15.3:0/1990539960 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+579 (secure 0 0 0) 0x7fd71c0a9d00 con 0x7fd724107700 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2743706157 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/174548513 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/174548513 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/3412581798 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:45.759 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/615172825 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4027937918 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/3412581798 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1221768244 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3389888238 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:45.760 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3812286739 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd70c0b2880 msgr2=0x7fd70c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd70c0b2880 0x7fd70c0b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd714005950 tx=0x7fd71400a300 comp rx=0 tx=0).stop 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 msgr2=0x7fd7241992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 0x7fd7241992a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd71c007ee0 tx=0x7fd71c00d3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 shutdown_connections 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd70c0b2880 0x7fd70c0b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd724107700 0x7fd7241992a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd724102620 0x7fd724198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 --2- 172.21.15.3:0/1990539960 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd724102fd0 0x7fd724198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.761+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 >> 172.21.15.3:0/1990539960 conn(0x7fd724079440 msgr2=0x7fd724100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.762+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 shutdown_connections 2024-05-08T05:50:45.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:45.762+0000 7fd729f4e700 1 -- 172.21.15.3:0/1990539960 wait complete. 2024-05-08T05:50:45.763 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2024-05-08T05:50:46.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:46 smithi003 ceph-mon[30137]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 374 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:46.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:46 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1990539960' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:50:46.475 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:50:46.475 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:50:46.495 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T05:50:46.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:46 smithi089 ceph-mon[34495]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 374 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:46.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:46 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1990539960' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:50:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:46 smithi104 ceph-mon[34559]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 374 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:50:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:46 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1990539960' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:50:46.791 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:48.537 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:48 smithi003 ceph-mon[30137]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:48.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/2389994182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7406bbd0 msgr2=0x7f2e7406c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/2389994182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7406bbd0 0x7f2e7406c050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2e64009990 tx=0x7f2e64009ca0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/2389994182 shutdown_connections 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/2389994182 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e7406c590 0x7f2e74110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/2389994182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e74109510 0x7f2e741098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/2389994182 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e7406bbd0 0x7f2e7406c050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/2389994182 >> 172.21.15.3:0/2389994182 conn(0x7f2e74079350 msgr2=0x7f2e74079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/2389994182 shutdown_connections 2024-05-08T05:50:48.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.546+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/2389994182 wait complete. 2024-05-08T05:50:48.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 Processor -- start 2024-05-08T05:50:48.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 -- start start 2024-05-08T05:50:48.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e7406bbd0 0x7f2e7419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e7406c590 0x7f2e7419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 0x7f2e741999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74113660 con 0x7f2e74109510 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741134e0 con 0x7f2e7406bbd0 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7bf0f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e741137e0 con 0x7f2e7406c590 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7a4ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 0x7f2e741999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7a4ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 0x7f2e741999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36814/0 (socket says 172.21.15.3:36814) 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7a4ac700 1 -- 172.21.15.3:0/3008051118 learned_addr learned my addr 172.21.15.3:0/3008051118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e79cab700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e7406bbd0 0x7f2e7419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e794aa700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e7406c590 0x7f2e7419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7a4ac700 1 -- 172.21.15.3:0/3008051118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e7406c590 msgr2=0x7f2e7419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.547+0000 7f2e7a4ac700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e7406c590 0x7f2e7419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e7a4ac700 1 -- 172.21.15.3:0/3008051118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e7406bbd0 msgr2=0x7f2e7419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e7a4ac700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e7406bbd0 0x7f2e7419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e7a4ac700 1 -- 172.21.15.3:0/3008051118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7419a120 con 0x7f2e74109510 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e7a4ac700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 0x7f2e741999c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c007ee0 tx=0x7f2e6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c004bb0 con 0x7f2e74109510 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7419a410 con 0x7f2e74109510 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6c004d50 con 0x7f2e74109510 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c005920 con 0x7f2e74109510 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.548+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e741a66d0 con 0x7f2e74109510 2024-05-08T05:50:48.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.549+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7410b9f0 con 0x7f2e74109510 2024-05-08T05:50:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.551+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f2e6c01a020 con 0x7f2e74109510 2024-05-08T05:50:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.552+0000 7f2e6affd700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2e600b2880 0x7f2e600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.552+0000 7f2e79cab700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2e600b2880 0x7f2e600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.552+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f2e6c0e2240 con 0x7f2e74109510 2024-05-08T05:50:48.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.552+0000 7f2e79cab700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2e600b2880 0x7f2e600b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e7419ab00 tx=0x7f2e7000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:48.555 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.555+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e6c010070 con 0x7f2e74109510 2024-05-08T05:50:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:48 smithi089 ceph-mon[34495]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:48 smithi104 ceph-mon[34559]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:48.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.790+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e7419aef0 con 0x7f2e74109510 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.791+0000 7f2e6affd700 1 -- 172.21.15.3:0/3008051118 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) ==== 81+0+579 (secure 0 0 0) 0x7f2e6c0a9c00 con 0x7f2e74109510 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2743706157 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:48.791 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/174548513 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/174548513 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/3412581798 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/615172825 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4027937918 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/3412581798 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1221768244 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3389888238 2024-05-09T05:42:06.081817+0000 2024-05-08T05:50:48.792 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3812286739 2024-05-09T05:42:33.122111+0000 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2e600b2880 msgr2=0x7f2e600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2e600b2880 0x7f2e600b4d40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2e7419ab00 tx=0x7f2e7000b3c0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 msgr2=0x7f2e741999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 0x7f2e741999c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c007ee0 tx=0x7f2e6c00d3b0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 shutdown_connections 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2e600b2880 0x7f2e600b4d40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2e74109510 0x7f2e741999c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2e7406bbd0 0x7f2e7419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 --2- 172.21.15.3:0/3008051118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e7406c590 0x7f2e7419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:48.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 >> 172.21.15.3:0/3008051118 conn(0x7f2e74079350 msgr2=0x7f2e74101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 shutdown_connections 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:48.793+0000 7f2e7bf0f700 1 -- 172.21.15.3:0/3008051118 wait complete. 2024-05-08T05:50:48.795 INFO:teuthology.orchestra.run.smithi003.stderr:listed 10 entries 2024-05-08T05:50:49.483 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:49 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3008051118' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:50:49.517 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi003.front.sepia.ceph.com... 2024-05-08T05:50:49.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:49.518 DEBUG:teuthology.orchestra.run.smithi003:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:50:49.548 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi003.stderr:ceph-fuse[76652]: fuse finished with error 0 and tester_r 0 2024-05-08T05:50:49.551 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:50:49.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:50:49.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:50:49.580 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T05:50:49.644 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:50:49.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:49 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3008051118' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:50:49.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:49 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3008051118' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:50:49.940 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:50.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:50 smithi003 ceph-mon[30137]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:50.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:50 smithi089 ceph-mon[34495]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:50 smithi104 ceph-mon[34559]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 -- 172.21.15.3:0/4024322074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514103750 msgr2=0x7f1514103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 --2- 172.21.15.3:0/4024322074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514103750 0x7f1514103b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f15080099c0 tx=0x7f1508009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 -- 172.21.15.3:0/4024322074 shutdown_connections 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 --2- 172.21.15.3:0/4024322074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1514108850 0x7f15141103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 --2- 172.21.15.3:0/4024322074 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514103750 0x7f1514103b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 --2- 172.21.15.3:0/4024322074 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1514104100 0x7f1514108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.660+0000 7f151abd0700 1 -- 172.21.15.3:0/4024322074 >> 172.21.15.3:0/4024322074 conn(0x7f151407a600 msgr2=0x7f151407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 -- 172.21.15.3:0/4024322074 shutdown_connections 2024-05-08T05:50:51.663 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 -- 172.21.15.3:0/4024322074 wait complete. 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 Processor -- start 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 -- start start 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1514103750 0x7f151419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1514104100 0x7f151419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 0x7f1514199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1514113710 con 0x7f1514108850 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1514113590 con 0x7f1514104100 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.661+0000 7f151abd0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1514113890 con 0x7f1514103750 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 0x7f1514199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 0x7f1514199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36834/0 (socket says 172.21.15.3:36834) 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 -- 172.21.15.3:0/3930203693 learned_addr learned my addr 172.21.15.3:0/3930203693 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151896c700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1514103750 0x7f151419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f1513fff700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1514104100 0x7f151419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 -- 172.21.15.3:0/3930203693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1514103750 msgr2=0x7f151419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1514103750 0x7f151419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 -- 172.21.15.3:0/3930203693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1514104100 msgr2=0x7f151419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1514104100 0x7f151419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 -- 172.21.15.3:0/3930203693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151419a170 con 0x7f1514108850 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151916d700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 0x7f1514199a10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f150c00b970 tx=0x7f150c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00d860 con 0x7f1514108850 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f150c00e430 con 0x7f1514108850 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151419a460 con 0x7f1514108850 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.662+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c017660 con 0x7f1514108850 2024-05-08T05:50:51.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.663+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141a6720 con 0x7f1514108850 2024-05-08T05:50:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.664+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151410ba90 con 0x7f1514108850 2024-05-08T05:50:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.665+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f150c00da00 con 0x7f1514108850 2024-05-08T05:50:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.666+0000 7f1511ffb700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f15040b2880 0x7f15040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.666+0000 7f151896c700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f15040b2880 0x7f15040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.666+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f150c0e21b0 con 0x7f1514108850 2024-05-08T05:50:51.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.666+0000 7f151896c700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f15040b2880 0x7f15040b4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f151419ab50 tx=0x7f150800b870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:51.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.671+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f150c013030 con 0x7f1514108850 2024-05-08T05:50:51.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.885+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f151410b470 con 0x7f1514108850 2024-05-08T05:50:51.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.887+0000 7f1511ffb700 1 -- 172.21.15.3:0/3930203693 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24572 (secure 0 0 0) 0x7f150c0a9b10 con 0x7f1514108850 2024-05-08T05:50:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:51.887 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:50:34.293465+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:51.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f15040b2880 msgr2=0x7f15040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f15040b2880 0x7f15040b4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f151419ab50 tx=0x7f150800b870 comp rx=0 tx=0).stop 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 msgr2=0x7f1514199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 0x7f1514199a10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f150c00b970 tx=0x7f150c00bc80 comp rx=0 tx=0).stop 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 shutdown_connections 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f15040b2880 0x7f15040b4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1514108850 0x7f1514199a10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1514103750 0x7f151419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 --2- 172.21.15.3:0/3930203693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1514104100 0x7f151419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 >> 172.21.15.3:0/3930203693 conn(0x7f151407a600 msgr2=0x7f1514101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 shutdown_connections 2024-05-08T05:50:51.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:51.889+0000 7f151abd0700 1 -- 172.21.15.3:0/3930203693 wait complete. 2024-05-08T05:50:52.614 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:52 smithi003 ceph-mon[30137]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:52.614 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:52 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3930203693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.614 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:52 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.646 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:50:52.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:52 smithi089 ceph-mon[34495]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:52 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3930203693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:52 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:52 smithi104 ceph-mon[34559]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:52 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3930203693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:52 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:50:53.510 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:54.641 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:54 smithi003 ceph-mon[30137]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:54.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.642 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:54 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:54 smithi089 ceph-mon[34495]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:54.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.765 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:54 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:54 smithi104 ceph-mon[34559]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-05-08T05:50:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:54 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 -- 172.21.15.3:0/3769170564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4103780 msgr2=0x7f03a4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 --2- 172.21.15.3:0/3769170564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4103780 0x7f03a4103b60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03980099c0 tx=0x7f0398009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 -- 172.21.15.3:0/3769170564 shutdown_connections 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 --2- 172.21.15.3:0/3769170564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a4108870 0x7f03a41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 --2- 172.21.15.3:0/3769170564 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4103780 0x7f03a4103b60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 --2- 172.21.15.3:0/3769170564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03a4104130 0x7f03a4108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 -- 172.21.15.3:0/3769170564 >> 172.21.15.3:0/3769170564 conn(0x7f03a407a610 msgr2=0x7f03a407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 -- 172.21.15.3:0/3769170564 shutdown_connections 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.217+0000 7f03aae3c700 1 -- 172.21.15.3:0/3769170564 wait complete. 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 Processor -- start 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 -- start start 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03a4103780 0x7f03a40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 0x7f03a4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a4108870 0x7f03a4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f03a4113640 con 0x7f03a4104130 2024-05-08T05:50:55.220 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41134c0 con 0x7f03a4108870 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03aae3c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03a41137c0 con 0x7f03a4103780 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 0x7f03a4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 0x7f03a4075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48000/0 (socket says 172.21.15.3:48000) 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 -- 172.21.15.3:0/1985668560 learned_addr learned my addr 172.21.15.3:0/1985668560 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a8bd8700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03a4103780 0x7f03a40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a93d9700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a4108870 0x7f03a4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 -- 172.21.15.3:0/1985668560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03a4103780 msgr2=0x7f03a40757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03a4103780 0x7f03a40757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 -- 172.21.15.3:0/1985668560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a4108870 msgr2=0x7f03a4076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a4108870 0x7f03a4076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.218+0000 7f03a3fff700 1 -- 172.21.15.3:0/1985668560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a41a7fc0 con 0x7f03a4104130 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.219+0000 7f03a3fff700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 0x7f03a4075ce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f039000d950 tx=0x7f039000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:55.221 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.219+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0390016070 con 0x7f03a4104130 2024-05-08T05:50:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.219+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a41a82b0 con 0x7f03a4104130 2024-05-08T05:50:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.219+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f039000fd40 con 0x7f03a4104130 2024-05-08T05:50:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.219+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039000e3b0 con 0x7f03a4104130 2024-05-08T05:50:55.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.219+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a41a8840 con 0x7f03a4104130 2024-05-08T05:50:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.220+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a410bab0 con 0x7f03a4104130 2024-05-08T05:50:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.221+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f0390019070 con 0x7f03a4104130 2024-05-08T05:50:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.222+0000 7f03a1ffb700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f03940b2880 0x7f03940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.222+0000 7f03a8bd8700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f03940b2880 0x7f03940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.222+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f03900e2060 con 0x7f03a4104130 2024-05-08T05:50:55.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.222+0000 7f03a8bd8700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f03940b2880 0x7f03940b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0398009990 tx=0x7f0398005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:55.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.227+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f03900a9a20 con 0x7f03a4104130 2024-05-08T05:50:55.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.469+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03a410b490 con 0x7f03a4104130 2024-05-08T05:50:55.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.471+0000 7f03a1ffb700 1 -- 172.21.15.3:0/1985668560 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24572 (secure 0 0 0) 0x7f03900a9320 con 0x7f03a4104130 2024-05-08T05:50:55.471 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:50:55.471 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:50:34.293465+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:50:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f03940b2880 msgr2=0x7f03940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f03940b2880 0x7f03940b4d40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0398009990 tx=0x7f0398005c40 comp rx=0 tx=0).stop 2024-05-08T05:50:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 msgr2=0x7f03a4075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 0x7f03a4075ce0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f039000d950 tx=0x7f039000dc60 comp rx=0 tx=0).stop 2024-05-08T05:50:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 shutdown_connections 2024-05-08T05:50:55.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f03a4108870 0x7f03a4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03a4103780 0x7f03a40757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f03940b2880 0x7f03940b4d40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 --2- 172.21.15.3:0/1985668560 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f03a4104130 0x7f03a4075ce0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:55.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 >> 172.21.15.3:0/1985668560 conn(0x7f03a407a610 msgr2=0x7f03a4101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:55.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 shutdown_connections 2024-05-08T05:50:55.475 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:55.474+0000 7f03aae3c700 1 -- 172.21.15.3:0/1985668560 wait complete. 2024-05-08T05:50:56.354 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:50:56.616 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:50:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:56 smithi003 ceph-mon[30137]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T05:50:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:56 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:56 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1985668560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:56.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:56 smithi089 ceph-mon[34495]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T05:50:56.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:56.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:56 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:56.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:56 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1985668560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:56 smithi104 ceph-mon[34559]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T05:50:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:56 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:56 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1985668560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:50:57.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:57.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:57.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:57.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:57.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:57.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:57.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:50:57.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:50:57.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.396+0000 7f7fdeacd700 1 -- 172.21.15.3:0/4230148164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103540 msgr2=0x7f7fd81039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.396+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/4230148164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103540 0x7f7fd81039c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00099c0 tx=0x7f7fc0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 -- 172.21.15.3:0/4230148164 shutdown_connections 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/4230148164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fd8103f00 0x7f7fd81103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/4230148164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 0x7f7fd81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/4230148164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103540 0x7f7fd81039c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.399 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 -- 172.21.15.3:0/4230148164 >> 172.21.15.3:0/4230148164 conn(0x7f7fd807a650 msgr2=0x7f7fd807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 -- 172.21.15.3:0/4230148164 shutdown_connections 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 -- 172.21.15.3:0/4230148164 wait complete. 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 Processor -- start 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 -- start start 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.397+0000 7f7fdeacd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fd8103540 0x7f7fd819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdeacd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 0x7f7fd819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdeacd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 0x7f7fd81a0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdeacd700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8113700 con 0x7f7fd8103f00 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdeacd700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8113580 con 0x7f7fd81095e0 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdeacd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd8113880 con 0x7f7fd8103540 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fd7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 0x7f7fd819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fd7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 0x7f7fd819c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48024/0 (socket says 172.21.15.3:48024) 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fd7fff700 1 -- 172.21.15.3:0/697254424 learned_addr learned my addr 172.21.15.3:0/697254424 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 0x7f7fd81a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.400 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdc869700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fd8103540 0x7f7fd819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 -- 172.21.15.3:0/697254424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fd8103540 msgr2=0x7f7fd819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fd8103540 0x7f7fd819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 -- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 msgr2=0x7f7fd819c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 0x7f7fd819c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 -- 172.21.15.3:0/697254424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd81a0e40 con 0x7f7fd81095e0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fd7fff700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 0x7f7fd819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdd06a700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 0x7f7fd81a0700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d720 tx=0x7f7fd000da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0016070 con 0x7f7fd81095e0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.398+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd81a1130 con 0x7f7fd81095e0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.399+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd000f690 con 0x7f7fd81095e0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.399+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0003a40 con 0x7f7fd81095e0 2024-05-08T05:50:58.401 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.399+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd81a89b0 con 0x7f7fd81095e0 2024-05-08T05:50:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.400+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd8104c50 con 0x7f7fd81095e0 2024-05-08T05:50:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.401+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f7fd0003be0 con 0x7f7fd81095e0 2024-05-08T05:50:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.402+0000 7f7fd5ffb700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7fc40b2900 0x7f7fc40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:50:58.402 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.402+0000 7f7fdc869700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7fc40b2900 0x7f7fc40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:50:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.402+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f7fd00e2040 con 0x7f7fd81095e0 2024-05-08T05:50:58.403 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.403+0000 7f7fdc869700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7fc40b2900 0x7f7fc40b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009550 tx=0x7f7fc80091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:50:58.406 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.406+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7fd001f070 con 0x7f7fd81095e0 2024-05-08T05:50:58.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:58 smithi003 ceph-mon[30137]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:50:58.667 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:58.667+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7fd81a8be0 con 0x7f7fd81095e0 2024-05-08T05:50:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:58 smithi089 ceph-mon[34495]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:50:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:58 smithi104 ceph-mon[34559]: pgmap v329: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:50:59.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.542+0000 7f7fd5ffb700 1 -- 172.21.15.3:0/697254424 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f7fd00a99a0 con 0x7f7fd81095e0 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7fc40b2900 msgr2=0x7f7fc40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7fc40b2900 0x7f7fc40b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009550 tx=0x7f7fc80091b0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 msgr2=0x7f7fd81a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 0x7f7fd81a0700 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000d720 tx=0x7f7fd000da30 comp rx=0 tx=0).stop 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 shutdown_connections 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7fd81095e0 0x7f7fd81a0700 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7fd8103540 0x7f7fd819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7fc40b2900 0x7f7fc40b4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.545+0000 7f7fdeacd700 1 --2- 172.21.15.3:0/697254424 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7fd8103f00 0x7f7fd819c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.546+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 >> 172.21.15.3:0/697254424 conn(0x7f7fd807a650 msgr2=0x7f7fd8101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:50:59.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.546+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 shutdown_connections 2024-05-08T05:50:59.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:50:59.546+0000 7f7fdeacd700 1 -- 172.21.15.3:0/697254424 wait complete. 2024-05-08T05:50:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:59 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/697254424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:50:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:50:59 smithi003 ceph-mon[30137]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:59 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/697254424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:50:59 smithi089 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:59 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/697254424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:50:59 smithi104 ceph-mon[34559]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:51:00.264 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-05-08T05:51:00.265 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-08T05:51:00.527 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:00.883 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:00 smithi003 ceph-mon[30137]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:51:00.883 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:00 smithi003 ceph-mon[30137]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:51:00.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:00 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:51:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:00 smithi089 ceph-mon[34495]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:51:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:00 smithi089 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:51:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:00 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:51:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:00 smithi104 ceph-mon[34559]: pgmap v330: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:51:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:00 smithi104 ceph-mon[34559]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:51:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:00 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:51:02.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/279601511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 msgr2=0x7fe7d01102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/279601511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01102c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b0d0 tx=0x7fe7c800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/279601511 shutdown_connections 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/279601511 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01102c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/279601511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7d0103610 0x7fe7d01039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/279601511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0103fc0 0x7fe7d0108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/279601511 >> 172.21.15.3:0/279601511 conn(0x7fe7d00feec0 msgr2=0x7fe7d01012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/279601511 shutdown_connections 2024-05-08T05:51:02.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.201+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/279601511 wait complete. 2024-05-08T05:51:02.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 Processor -- start 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 -- start start 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7d0103610 0x7fe7d019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0103fc0 0x7fe7d019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113600 con 0x7fe7d01086f0 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113480 con 0x7fe7d0103610 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.202+0000 7fe7d6d9f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0113780 con 0x7fe7d0103fc0 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d559c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0103fc0 0x7fe7d019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48048/0 (socket says 172.21.15.3:48048) 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 -- 172.21.15.3:0/2254359196 learned_addr learned my addr 172.21.15.3:0/2254359196 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d5d9d700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7d0103610 0x7fe7d019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 -- 172.21.15.3:0/2254359196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0103fc0 msgr2=0x7fe7d019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0103fc0 0x7fe7d019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 -- 172.21.15.3:0/2254359196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7d0103610 msgr2=0x7fe7d019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7d0103610 0x7fe7d019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 -- 172.21.15.3:0/2254359196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d019a040 con 0x7fe7d01086f0 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d659e700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01998e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c801f070 con 0x7fe7d01086f0 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.203+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d019a2d0 con 0x7fe7d01086f0 2024-05-08T05:51:02.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.204+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d00756d0 con 0x7fe7d01086f0 2024-05-08T05:51:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.204+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c8007e50 con 0x7fe7d01086f0 2024-05-08T05:51:02.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.204+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8029060 con 0x7fe7d01086f0 2024-05-08T05:51:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.205+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d0104af0 con 0x7fe7d01086f0 2024-05-08T05:51:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.206+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7fe7c8022070 con 0x7fe7d01086f0 2024-05-08T05:51:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.207+0000 7fe7c6ffd700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe7bc0b2900 0x7fe7bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.207+0000 7fe7d5d9d700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe7bc0b2900 0x7fe7bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.207+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fe7c80e3210 con 0x7fe7d01086f0 2024-05-08T05:51:02.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.208+0000 7fe7d5d9d700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe7bc0b2900 0x7fe7bc0b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe7d01084a0 tx=0x7fe7cc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:02.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.212+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe7c80aac60 con 0x7fe7d01086f0 2024-05-08T05:51:02.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.476+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe7d019adc0 con 0x7fe7d01086f0 2024-05-08T05:51:02.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.544+0000 7fe7c6ffd700 1 -- 172.21.15.3:0/2254359196 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fe7c80aa560 con 0x7fe7d01086f0 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe7bc0b2900 msgr2=0x7fe7bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe7bc0b2900 0x7fe7bc0b4dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe7d01084a0 tx=0x7fe7cc00a380 comp rx=0 tx=0).stop 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 msgr2=0x7fe7d01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01998e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009fd0 tx=0x7fe7c8007a20 comp rx=0 tx=0).stop 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 shutdown_connections 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe7bc0b2900 0x7fe7bc0b4dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe7d01086f0 0x7fe7d01998e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe7d0103610 0x7fe7d019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 --2- 172.21.15.3:0/2254359196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7d0103fc0 0x7fe7d019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 >> 172.21.15.3:0/2254359196 conn(0x7fe7d00feec0 msgr2=0x7fe7d00ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 shutdown_connections 2024-05-08T05:51:02.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:02.546+0000 7fe7d6d9f700 1 -- 172.21.15.3:0/2254359196 wait complete. 2024-05-08T05:51:02.844 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:02 smithi003 ceph-mon[30137]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:51:02.844 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:02 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2254359196' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:02 smithi089 ceph-mon[34495]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:51:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:02 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2254359196' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:02 smithi104 ceph-mon[34559]: pgmap v331: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:51:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:02 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2254359196' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:51:03.240 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-08T05:51:03.499 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:03.873 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:03 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2254359196' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:51:03.873 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:03 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:51:03.873 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:03 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:51:04.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:03 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2254359196' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:51:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:03 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:51:04.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:03 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:51:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:03 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2254359196' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:51:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:03 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:51:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:03 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:51:04.825 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:04 smithi003 ceph-mon[30137]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2024-05-08T05:51:05.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:04 smithi089 ceph-mon[34495]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2024-05-08T05:51:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:04 smithi104 ceph-mon[34559]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 -- 172.21.15.3:0/2527397557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb4103f00 msgr2=0x7febb41103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 --2- 172.21.15.3:0/2527397557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb4103f00 0x7febb41103b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7febac0099c0 tx=0x7febac009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 -- 172.21.15.3:0/2527397557 shutdown_connections 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 --2- 172.21.15.3:0/2527397557 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb4103f00 0x7febb41103b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 --2- 172.21.15.3:0/2527397557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febb41095e0 0x7febb41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 --2- 172.21.15.3:0/2527397557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febb4103540 0x7febb41039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 -- 172.21.15.3:0/2527397557 >> 172.21.15.3:0/2527397557 conn(0x7febb407a650 msgr2=0x7febb407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 -- 172.21.15.3:0/2527397557 shutdown_connections 2024-05-08T05:51:05.216 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.213+0000 7febbc269700 1 -- 172.21.15.3:0/2527397557 wait complete. 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 Processor -- start 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 -- start start 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febb4103540 0x7febb419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febb4103f00 0x7febb419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 0x7febb41a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7febb41137c0 con 0x7febb41095e0 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7febb4113640 con 0x7febb4103f00 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febbc269700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7febb4113940 con 0x7febb4103540 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febba806700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 0x7febb41a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febba806700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 0x7febb41a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57770/0 (socket says 172.21.15.3:57770) 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.214+0000 7febba806700 1 -- 172.21.15.3:0/1774710919 learned_addr learned my addr 172.21.15.3:0/1774710919 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba005700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febb4103540 0x7febb419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febb9804700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febb4103f00 0x7febb419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.217 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba806700 1 -- 172.21.15.3:0/1774710919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febb4103540 msgr2=0x7febb419bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba806700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febb4103540 0x7febb419bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba806700 1 -- 172.21.15.3:0/1774710919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febb4103f00 msgr2=0x7febb419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba806700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febb4103f00 0x7febb419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba806700 1 -- 172.21.15.3:0/1774710919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb41a0da0 con 0x7febb41095e0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febba806700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 0x7febb41a0660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febac009990 tx=0x7febac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac027070 con 0x7febb41095e0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb41a1030 con 0x7febb41095e0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febac004150 con 0x7febb41095e0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac00f460 con 0x7febb41095e0 2024-05-08T05:51:05.218 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.215+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb41a8910 con 0x7febb41095e0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.216+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb4104c50 con 0x7febb41095e0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.218+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7febac02a070 con 0x7febb41095e0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.218+0000 7febb37fe700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7feb9c0b2880 0x7feb9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.218+0000 7febba005700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7feb9c0b2880 0x7feb9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.219+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7febac0e3b60 con 0x7febb41095e0 2024-05-08T05:51:05.219 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.219+0000 7febba005700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7feb9c0b2880 0x7feb9c0b4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feba4009f20 tx=0x7feba40094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:05.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.222+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febac031070 con 0x7febb41095e0 2024-05-08T05:51:05.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.492+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7febb41a8ae0 con 0x7febb41095e0 2024-05-08T05:51:05.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.568+0000 7febb37fe700 1 -- 172.21.15.3:0/1774710919 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7febac013070 con 0x7febb41095e0 2024-05-08T05:51:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7feb9c0b2880 msgr2=0x7feb9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7feb9c0b2880 0x7feb9c0b4d40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feba4009f20 tx=0x7feba40094f0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 msgr2=0x7febb41a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 0x7febb41a0660 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7febac009990 tx=0x7febac0047b0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 shutdown_connections 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7feb9c0b2880 0x7feb9c0b4d40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7febb41095e0 0x7febb41a0660 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febb4103540 0x7febb419bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 --2- 172.21.15.3:0/1774710919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7febb4103f00 0x7febb419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 >> 172.21.15.3:0/1774710919 conn(0x7febb407a650 msgr2=0x7febb410e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 shutdown_connections 2024-05-08T05:51:05.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:05.571+0000 7febbc269700 1 -- 172.21.15.3:0/1774710919 wait complete. 2024-05-08T05:51:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:05 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1774710919' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:06.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:05 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1774710919' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:06.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:05 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1774710919' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:51:06.303 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-05-08T05:51:06.303 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-08T05:51:06.563 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:06.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:06 smithi003 ceph-mon[30137]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 0 op/s 2024-05-08T05:51:06.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:06 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1774710919' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:51:06.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:06 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:51:07.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:06 smithi089 ceph-mon[34495]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 0 op/s 2024-05-08T05:51:07.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:06 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1774710919' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:51:07.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:06 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:51:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:06 smithi104 ceph-mon[34559]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 0 op/s 2024-05-08T05:51:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:06 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1774710919' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:51:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:06 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:51:08.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 -- 172.21.15.3:0/4057330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb4102fe0 msgr2=0x7f4cb4106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 --2- 172.21.15.3:0/4057330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb4102fe0 0x7f4cb4106fe0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0099c0 tx=0x7f4c9c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 -- 172.21.15.3:0/4057330635 shutdown_connections 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 --2- 172.21.15.3:0/4057330635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cb41076c0 0x7f4cb410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 --2- 172.21.15.3:0/4057330635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102630 0x7f4cb4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 --2- 172.21.15.3:0/4057330635 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb4102fe0 0x7f4cb4106fe0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 -- 172.21.15.3:0/4057330635 >> 172.21.15.3:0/4057330635 conn(0x7f4cb4079350 msgr2=0x7f4cb4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.349+0000 7f4cb9560700 1 -- 172.21.15.3:0/4057330635 shutdown_connections 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.350+0000 7f4cb9560700 1 -- 172.21.15.3:0/4057330635 wait complete. 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.350+0000 7f4cb9560700 1 Processor -- start 2024-05-08T05:51:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 -- start start 2024-05-08T05:51:08.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cb4102630 0x7f4cb419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102fe0 0x7f4cb419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 0x7f4cb4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb4112550 con 0x7f4cb41076c0 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41123d0 con 0x7f4cb4102fe0 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb9560700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cb41126d0 con 0x7f4cb4102630 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 0x7f4cb4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb2ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cb4102630 0x7f4cb419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 0x7f4cb4198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57788/0 (socket says 172.21.15.3:57788) 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 -- 172.21.15.3:0/857479947 learned_addr learned my addr 172.21.15.3:0/857479947 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4caa7fc700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102fe0 0x7f4cb419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 -- 172.21.15.3:0/857479947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cb4102630 msgr2=0x7f4cb419e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cb4102630 0x7f4cb419e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 -- 172.21.15.3:0/857479947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102fe0 msgr2=0x7f4cb419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102fe0 0x7f4cb419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4cb37fe700 1 -- 172.21.15.3:0/857479947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb41990c0 con 0x7f4cb41076c0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.351+0000 7f4caa7fc700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102fe0 0x7f4cb419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.352+0000 7f4cb37fe700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 0x7f4cb4198960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00ada0 tx=0x7f4cac00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.352+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac017020 con 0x7f4cb41076c0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.352+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cb41993b0 con 0x7f4cb41076c0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.352+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cac00ce30 con 0x7f4cb41076c0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.352+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac005350 con 0x7f4cb41076c0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.352+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cb41a5670 con 0x7f4cb41076c0 2024-05-08T05:51:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.353+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb406a190 con 0x7f4cb41076c0 2024-05-08T05:51:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.354+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 16) ==== 149651+0+0 (secure 0 0 0) 0x7f4cac00c980 con 0x7f4cb41076c0 2024-05-08T05:51:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.355+0000 7f4cb0ff9700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f4ca00b2880 0x7f4ca00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.355+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f4cac0e1c40 con 0x7f4cb41076c0 2024-05-08T05:51:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.355+0000 7f4cb2ffd700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f4ca00b2880 0x7f4ca00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.356+0000 7f4cb2ffd700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f4ca00b2880 0x7f4ca00b4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4005950 tx=0x7f4ca400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:08.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.360+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4cac021020 con 0x7f4cb41076c0 2024-05-08T05:51:08.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.611+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4cb406bc30 con 0x7f4ca00b2880 2024-05-08T05:51:08.612 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:08 smithi003 ceph-mon[30137]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 0 op/s 2024-05-08T05:51:08.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.632+0000 7f4cb0ff9700 1 -- 172.21.15.3:0/857479947 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4cb406bc30 con 0x7f4ca00b2880 2024-05-08T05:51:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f4ca00b2880 msgr2=0x7f4ca00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f4ca00b2880 0x7f4ca00b4d40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4005950 tx=0x7f4ca400a300 comp rx=0 tx=0).stop 2024-05-08T05:51:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 msgr2=0x7f4cb4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 0x7f4cb4198960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00ada0 tx=0x7f4cac00c330 comp rx=0 tx=0).stop 2024-05-08T05:51:08.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 shutdown_connections 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f4ca00b2880 0x7f4ca00b4d40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4cb41076c0 0x7f4cb4198960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cb4102630 0x7f4cb419e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 --2- 172.21.15.3:0/857479947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4cb4102fe0 0x7f4cb419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 >> 172.21.15.3:0/857479947 conn(0x7f4cb4079350 msgr2=0x7f4cb4100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 shutdown_connections 2024-05-08T05:51:08.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:08.635+0000 7f4cb9560700 1 -- 172.21.15.3:0/857479947 wait complete. 2024-05-08T05:51:08.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:51:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:51:08.617+0000 7f0a428fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:08.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:51:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:51:08.617+0000 7f0a428fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:08.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:51:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:51:08.617+0000 7f0a428fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:08.911 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:51:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:51:08.617+0000 7f0a428fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:08.911 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:51:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:51:08.617+0000 7f0a428fa700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:51:09.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:08 smithi089 ceph-mon[34495]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 0 op/s 2024-05-08T05:51:09.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:08 smithi104 ceph-mon[34559]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 0 op/s 2024-05-08T05:51:09.331 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T05:51:09.581 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:09 smithi003 ceph-mon[30137]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:09.591 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:10.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:09 smithi089 ceph-mon[34495]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:10.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:09 smithi104 ceph-mon[34559]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:10.713 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:10 smithi003 ceph-mon[30137]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:10.714 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:10 smithi003 ceph-mon[30137]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:51:11.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:10 smithi089 ceph-mon[34495]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:11.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:10 smithi089 ceph-mon[34495]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:51:11.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:10 smithi104 ceph-mon[34559]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:10 smithi104 ceph-mon[34559]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 -- 172.21.15.3:0/3150304503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c107420 msgr2=0x7fd53c107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 --2- 172.21.15.3:0/3150304503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c107420 0x7fd53c107800 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd53800b0a0 tx=0x7fd53800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 -- 172.21.15.3:0/3150304503 shutdown_connections 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 --2- 172.21.15.3:0/3150304503 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd53c1030b0 0x7fd53c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 --2- 172.21.15.3:0/3150304503 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c107420 0x7fd53c107800 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 --2- 172.21.15.3:0/3150304503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd53c1026f0 0x7fd53c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.255+0000 7fd543e27700 1 -- 172.21.15.3:0/3150304503 >> 172.21.15.3:0/3150304503 conn(0x7fd53c078210 msgr2=0x7fd53c078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 -- 172.21.15.3:0/3150304503 shutdown_connections 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 -- 172.21.15.3:0/3150304503 wait complete. 2024-05-08T05:51:11.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 Processor -- start 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 -- start start 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd53c1026f0 0x7fd53c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 0x7fd53c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd53c107420 0x7fd53c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c112650 con 0x7fd53c1030b0 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1124d0 con 0x7fd53c1026f0 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.256+0000 7fd543e27700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd53c1127d0 con 0x7fd53c107420 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 0x7fd53c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 0x7fd53c19fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57838/0 (socket says 172.21.15.3:57838) 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 -- 172.21.15.3:0/4024004992 learned_addr learned my addr 172.21.15.3:0/4024004992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd541bc3700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd53c1026f0 0x7fd53c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5423c4700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd53c107420 0x7fd53c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 -- 172.21.15.3:0/4024004992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd53c107420 msgr2=0x7fd53c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd53c107420 0x7fd53c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 -- 172.21.15.3:0/4024004992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd53c1026f0 msgr2=0x7fd53c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd53c1026f0 0x7fd53c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 -- 172.21.15.3:0/4024004992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53c19a230 con 0x7fd53c1030b0 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd5413c2700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 0x7fd53c19fe50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00b530 tx=0x7fd52c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c015020 con 0x7fd53c1030b0 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53c19a520 con 0x7fd53c1030b0 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd52c011ce0 con 0x7fd53c1030b0 2024-05-08T05:51:11.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.257+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52c010dd0 con 0x7fd53c1030b0 2024-05-08T05:51:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.258+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd53c1a67e0 con 0x7fd53c1030b0 2024-05-08T05:51:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.258+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd53c103b10 con 0x7fd53c1030b0 2024-05-08T05:51:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.260+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd52c018020 con 0x7fd53c1030b0 2024-05-08T05:51:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.261+0000 7fd532ffd700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd5280b2990 0x7fd5280b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:11.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.261+0000 7fd541bc3700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd5280b2990 0x7fd5280b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.261+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7fd52c0e2210 con 0x7fd53c1030b0 2024-05-08T05:51:11.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.261+0000 7fd541bc3700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd5280b2990 0x7fd5280b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd538009fd0 tx=0x7fd53800b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:11.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.265+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd52c01f020 con 0x7fd53c1030b0 2024-05-08T05:51:11.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.514+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd53c0638e0 con 0x7fd5280b2990 2024-05-08T05:51:11.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.543+0000 7fd532ffd700 1 -- 172.21.15.3:0/4024004992 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd53c0638e0 con 0x7fd5280b2990 2024-05-08T05:51:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.545+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd5280b2990 msgr2=0x7fd5280b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.545+0000 7fd543e27700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd5280b2990 0x7fd5280b4e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd538009fd0 tx=0x7fd53800b7b0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.545+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 msgr2=0x7fd53c19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.545+0000 7fd543e27700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 0x7fd53c19fe50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00b530 tx=0x7fd52c00b840 comp rx=0 tx=0).stop 2024-05-08T05:51:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.545+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 shutdown_connections 2024-05-08T05:51:11.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd53c107420 0x7fd53c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd53c1026f0 0x7fd53c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd5280b2990 0x7fd5280b4e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 --2- 172.21.15.3:0/4024004992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd53c1030b0 0x7fd53c19fe50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 >> 172.21.15.3:0/4024004992 conn(0x7fd53c078210 msgr2=0x7fd53c06bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 shutdown_connections 2024-05-08T05:51:11.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:11.546+0000 7fd543e27700 1 -- 172.21.15.3:0/4024004992 wait complete. 2024-05-08T05:51:12.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T05:51:12.518 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:12.865 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:12 smithi003 ceph-mon[30137]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:12.865 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:12 smithi003 ceph-mon[30137]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:12 smithi089 ceph-mon[34495]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:13.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:12 smithi089 ceph-mon[34495]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:12 smithi104 ceph-mon[34559]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:12 smithi104 ceph-mon[34559]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 -- 172.21.15.3:0/455669602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c1076e0 msgr2=0x7f608c10f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 --2- 172.21.15.3:0/455669602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c1076e0 0x7f608c10f2b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f608400b0d0 tx=0x7f608400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 -- 172.21.15.3:0/455669602 shutdown_connections 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 --2- 172.21.15.3:0/455669602 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c1076e0 0x7f608c10f2b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 --2- 172.21.15.3:0/455669602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f608c102690 0x7f608c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 --2- 172.21.15.3:0/455669602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c103040 0x7f608c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 -- 172.21.15.3:0/455669602 >> 172.21.15.3:0/455669602 conn(0x7f608c0783f0 msgr2=0x7f608c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.223 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 -- 172.21.15.3:0/455669602 shutdown_connections 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.220+0000 7f6092914700 1 -- 172.21.15.3:0/455669602 wait complete. 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 Processor -- start 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 -- start start 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c102690 0x7f608c1a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 0x7f608c1a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f608c1076e0 0x7f608c1a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1125a0 con 0x7f608c103040 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f608c112420 con 0x7f608c102690 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6092914700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f608c112720 con 0x7f608c1076e0 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6083fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 0x7f608c1a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.221+0000 7f6083fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 0x7f608c1a63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35246/0 (socket says 172.21.15.3:35246) 2024-05-08T05:51:14.224 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 -- 172.21.15.3:0/1932865482 learned_addr learned my addr 172.21.15.3:0/1932865482 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f608bfff700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c102690 0x7f608c1a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6090eb1700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f608c1076e0 0x7f608c1a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 -- 172.21.15.3:0/1932865482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f608c1076e0 msgr2=0x7f608c1a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f608c1076e0 0x7f608c1a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 -- 172.21.15.3:0/1932865482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c102690 msgr2=0x7f608c1a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c102690 0x7f608c1a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 -- 172.21.15.3:0/1932865482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c1a0b80 con 0x7f608c103040 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6083fff700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 0x7f608c1a63a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f607400d980 tx=0x7f607400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074017070 con 0x7f608c103040 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c1a0e10 con 0x7f608c103040 2024-05-08T05:51:14.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c1ace20 con 0x7f608c103040 2024-05-08T05:51:14.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6074009de0 con 0x7f608c103040 2024-05-08T05:51:14.226 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.222+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607400f3c0 con 0x7f608c103040 2024-05-08T05:51:14.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.224+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608c06a190 con 0x7f608c103040 2024-05-08T05:51:14.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.225+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f6074009930 con 0x7f608c103040 2024-05-08T05:51:14.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.226+0000 7f6089ffb700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60780b2990 0x7f60780b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:14.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.226+0000 7f608bfff700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60780b2990 0x7f60780b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:14.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.226+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f60740e24f0 con 0x7f608c103040 2024-05-08T05:51:14.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.226+0000 7f608bfff700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60780b2990 0x7f60780b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f607c005950 tx=0x7f607c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:14.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.230+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60740a9cf0 con 0x7f608c103040 2024-05-08T05:51:14.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.480+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f608c0638e0 con 0x7f60780b2990 2024-05-08T05:51:14.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.486+0000 7f6089ffb700 1 -- 172.21.15.3:0/1932865482 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f608c0638e0 con 0x7f60780b2990 2024-05-08T05:51:14.486 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_0/227fbf48-0c39-45af-a2df-48517ac55f87 2024-05-08T05:51:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60780b2990 msgr2=0x7f60780b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60780b2990 0x7f60780b4e50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f607c005950 tx=0x7f607c00a300 comp rx=0 tx=0).stop 2024-05-08T05:51:14.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 msgr2=0x7f608c1a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 0x7f608c1a63a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f607400d980 tx=0x7f607400dc90 comp rx=0 tx=0).stop 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 shutdown_connections 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f608c1076e0 0x7f608c1a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f608c102690 0x7f608c1a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60780b2990 0x7f60780b4e50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 --2- 172.21.15.3:0/1932865482 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f608c103040 0x7f608c1a63a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 >> 172.21.15.3:0/1932865482 conn(0x7f608c0783f0 msgr2=0x7f608c10e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 shutdown_connections 2024-05-08T05:51:14.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:14.488+0000 7f6092914700 1 -- 172.21.15.3:0/1932865482 wait complete. 2024-05-08T05:51:14.809 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:14 smithi003 ceph-mon[30137]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:51:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:14 smithi089 ceph-mon[34495]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:51:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:14 smithi104 ceph-mon[34559]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:51:15.178 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T05:51:15.434 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:15.821 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:15 smithi003 ceph-mon[30137]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:15 smithi089 ceph-mon[34495]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:15 smithi104 ceph-mon[34559]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:16.881 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:16 smithi003 ceph-mon[30137]: pgmap v338: 129 pgs: 129 active+clean; 606 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-08T05:51:16.913 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:16 smithi089 ceph-mon[34495]: pgmap v338: 129 pgs: 129 active+clean; 606 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-08T05:51:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:16 smithi104 ceph-mon[34559]: pgmap v338: 129 pgs: 129 active+clean; 606 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 -- 172.21.15.3:0/3867915913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef8102d60 msgr2=0x7efef810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 --2- 172.21.15.3:0/3867915913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef8102d60 0x7efef810f250 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efef000b0a0 tx=0x7efef000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 -- 172.21.15.3:0/3867915913 shutdown_connections 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 --2- 172.21.15.3:0/3867915913 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef8102d60 0x7efef810f250 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 --2- 172.21.15.3:0/3867915913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8108410 0x7efef81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 --2- 172.21.15.3:0/3867915913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efef81023a0 0x7efef8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 -- 172.21.15.3:0/3867915913 >> 172.21.15.3:0/3867915913 conn(0x7efef8079350 msgr2=0x7efef8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 -- 172.21.15.3:0/3867915913 shutdown_connections 2024-05-08T05:51:17.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.250+0000 7efefe7df700 1 -- 172.21.15.3:0/3867915913 wait complete. 2024-05-08T05:51:17.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 Processor -- start 2024-05-08T05:51:17.253 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 -- start start 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 0x7efef819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8102d60 0x7efef819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efef8108410 0x7efef8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efef8112530 con 0x7efef81023a0 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efef81123b0 con 0x7efef8102d60 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efefe7df700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efef81126b0 con 0x7efef8108410 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efef7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 0x7efef819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efef7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 0x7efef819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35254/0 (socket says 172.21.15.3:35254) 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.251+0000 7efef7fff700 1 -- 172.21.15.3:0/2859549017 learned_addr learned my addr 172.21.15.3:0/2859549017 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efefcd7c700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efef8108410 0x7efef8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efeeffff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8102d60 0x7efef819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef7fff700 1 -- 172.21.15.3:0/2859549017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efef8108410 msgr2=0x7efef8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.254 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef7fff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efef8108410 0x7efef8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef7fff700 1 -- 172.21.15.3:0/2859549017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8102d60 msgr2=0x7efef819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef7fff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8102d60 0x7efef819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef7fff700 1 -- 172.21.15.3:0/2859549017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef8199030 con 0x7efef81023a0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efeeffff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8102d60 0x7efef819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef7fff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 0x7efef819e680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efee8009970 tx=0x7efee8009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee801c070 con 0x7efef81023a0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef8199320 con 0x7efef81023a0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee800d260 con 0x7efef81023a0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8017a60 con 0x7efef81023a0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.252+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef806b180 con 0x7efef81023a0 2024-05-08T05:51:17.255 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.253+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef8103ac0 con 0x7efef81023a0 2024-05-08T05:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.255+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7efee801f070 con 0x7efef81023a0 2024-05-08T05:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.255+0000 7efef5ffb700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efee40b2910 0x7efee40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.255+0000 7efeeffff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efee40b2910 0x7efee40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.255+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7efee80e20d0 con 0x7efef81023a0 2024-05-08T05:51:17.256 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.256+0000 7efeeffff700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efee40b2910 0x7efee40b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efef8079950 tx=0x7efee000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:17.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.259+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efee80a98d0 con 0x7efef81023a0 2024-05-08T05:51:17.508 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.508+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efef806bc80 con 0x7efee40b2910 2024-05-08T05:51:17.533 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.533+0000 7efef5ffb700 1 -- 172.21.15.3:0/2859549017 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efef806bc80 con 0x7efee40b2910 2024-05-08T05:51:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efee40b2910 msgr2=0x7efee40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efee40b2910 0x7efee40b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efef8079950 tx=0x7efee000b5f0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 msgr2=0x7efef819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 0x7efef819e680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efee8009970 tx=0x7efee8009c80 comp rx=0 tx=0).stop 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 shutdown_connections 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efef8108410 0x7efef8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7efee40b2910 0x7efee40b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efef81023a0 0x7efef819e680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 --2- 172.21.15.3:0/2859549017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef8102d60 0x7efef819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 >> 172.21.15.3:0/2859549017 conn(0x7efef8079350 msgr2=0x7efef8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 shutdown_connections 2024-05-08T05:51:17.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:17.536+0000 7efefe7df700 1 -- 172.21.15.3:0/2859549017 wait complete. 2024-05-08T05:51:18.198 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T05:51:18.455 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:18.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:18 smithi003 ceph-mon[30137]: pgmap v339: 129 pgs: 129 active+clean; 606 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-08T05:51:18.831 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:18 smithi003 ceph-mon[30137]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:19.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:18 smithi089 ceph-mon[34495]: pgmap v339: 129 pgs: 129 active+clean; 606 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-08T05:51:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:18 smithi089 ceph-mon[34495]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:18 smithi104 ceph-mon[34559]: pgmap v339: 129 pgs: 129 active+clean; 606 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 3 op/s 2024-05-08T05:51:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:18 smithi104 ceph-mon[34559]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 -- 172.21.15.3:0/1026237693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020109590 msgr2=0x7f8020109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 --2- 172.21.15.3:0/1026237693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020109590 0x7f8020109970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f80100099c0 tx=0x7f8010009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 -- 172.21.15.3:0/1026237693 shutdown_connections 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 --2- 172.21.15.3:0/1026237693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f802006c630 0x7f8020110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 --2- 172.21.15.3:0/1026237693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8020109590 0x7f8020109970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 --2- 172.21.15.3:0/1026237693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802006bc70 0x7f802006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 -- 172.21.15.3:0/1026237693 >> 172.21.15.3:0/1026237693 conn(0x7f8020079440 msgr2=0x7f8020079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 -- 172.21.15.3:0/1026237693 shutdown_connections 2024-05-08T05:51:20.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.259+0000 7f802636d700 1 -- 172.21.15.3:0/1026237693 wait complete. 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 Processor -- start 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 -- start start 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802006bc70 0x7f8020102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 0x7f8020102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8020109590 0x7f8020102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8020113690 con 0x7f802006c630 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8020113510 con 0x7f802006bc70 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802636d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8020113810 con 0x7f8020109590 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 0x7f8020102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 0x7f8020102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35280/0 (socket says 172.21.15.3:35280) 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 -- 172.21.15.3:0/4068480488 learned_addr learned my addr 172.21.15.3:0/4068480488 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f801ffff700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802006bc70 0x7f8020102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f802490a700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8020109590 0x7f8020102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 -- 172.21.15.3:0/4068480488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8020109590 msgr2=0x7f8020102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8020109590 0x7f8020102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 -- 172.21.15.3:0/4068480488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802006bc70 msgr2=0x7f8020102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.260+0000 7f8017fff700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802006bc70 0x7f8020102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f8017fff700 1 -- 172.21.15.3:0/4068480488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80201a8070 con 0x7f802006c630 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f8017fff700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 0x7f8020102950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f800800c920 tx=0x7f800800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008004a30 con 0x7f802006c630 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80080054d0 con 0x7f802006c630 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80201a8360 con 0x7f802006c630 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008007c70 con 0x7f802006c630 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.261+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201a88f0 con 0x7f802006c630 2024-05-08T05:51:20.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.262+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802010ba20 con 0x7f802006c630 2024-05-08T05:51:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.264+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f8008004c60 con 0x7f802006c630 2024-05-08T05:51:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.264+0000 7f801dffb700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f800c0b2990 0x7f800c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.264+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f80080e25b0 con 0x7f802006c630 2024-05-08T05:51:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.264+0000 7f801ffff700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f800c0b2990 0x7f800c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:20.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.265+0000 7f801ffff700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f800c0b2990 0x7f800c0b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8010009990 tx=0x7f801000b790 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:20.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.268+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8008014030 con 0x7f802006c630 2024-05-08T05:51:20.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.518+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f8020103940 con 0x7f800c0b2990 2024-05-08T05:51:20.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.524+0000 7f801dffb700 1 -- 172.21.15.3:0/4068480488 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8020103940 con 0x7f800c0b2990 2024-05-08T05:51:20.524 INFO:teuthology.orchestra.run.smithi003.stdout:/volumes/qa/sv_1/95b7cec1-e786-453a-8829-7f21d652d298 2024-05-08T05:51:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f800c0b2990 msgr2=0x7f800c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f800c0b2990 0x7f800c0b4e50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8010009990 tx=0x7f801000b790 comp rx=0 tx=0).stop 2024-05-08T05:51:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 msgr2=0x7f8020102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 0x7f8020102950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f800800c920 tx=0x7f800800cc30 comp rx=0 tx=0).stop 2024-05-08T05:51:20.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 shutdown_connections 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8020109590 0x7f8020102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f802006bc70 0x7f8020102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f800c0b2990 0x7f800c0b4e50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 --2- 172.21.15.3:0/4068480488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f802006c630 0x7f8020102950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 >> 172.21.15.3:0/4068480488 conn(0x7f8020079440 msgr2=0x7f80201010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 shutdown_connections 2024-05-08T05:51:20.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:20.526+0000 7f802636d700 1 -- 172.21.15.3:0/4068480488 wait complete. 2024-05-08T05:51:20.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:20 smithi003 ceph-mon[30137]: pgmap v340: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-05-08T05:51:21.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:20 smithi089 ceph-mon[34495]: pgmap v340: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-05-08T05:51:21.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:20 smithi104 ceph-mon[34559]: pgmap v340: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.4 KiB/s wr, 5 op/s 2024-05-08T05:51:21.290 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:51:21.545 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:21.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:21 smithi003 ceph-mon[30137]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:21 smithi089 ceph-mon[34495]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:21 smithi104 ceph-mon[34559]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:51:22.877 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:22 smithi003 ceph-mon[30137]: pgmap v341: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-05-08T05:51:23.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:22 smithi089 ceph-mon[34495]: pgmap v341: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-05-08T05:51:23.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:22 smithi104 ceph-mon[34559]: pgmap v341: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.4 KiB/s wr, 4 op/s 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.270+0000 7f602dbc7700 1 -- 172.21.15.3:0/3290658188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 msgr2=0x7f602806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.270+0000 7f602dbc7700 1 --2- 172.21.15.3:0/3290658188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602806b600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f601c009990 tx=0x7f601c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 -- 172.21.15.3:0/3290658188 shutdown_connections 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 --2- 172.21.15.3:0/3290658188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f602806bb40 0x7f6028110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 --2- 172.21.15.3:0/3290658188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6028109510 0x7f60281098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 --2- 172.21.15.3:0/3290658188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602806b600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 -- 172.21.15.3:0/3290658188 >> 172.21.15.3:0/3290658188 conn(0x7f6028079350 msgr2=0x7f6028079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 -- 172.21.15.3:0/3290658188 shutdown_connections 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 -- 172.21.15.3:0/3290658188 wait complete. 2024-05-08T05:51:23.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 Processor -- start 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.271+0000 7f602dbc7700 1 -- start start 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f602dbc7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f602dbc7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f602806bb40 0x7f602819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f602dbc7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6028109510 0x7f6028199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f602dbc7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6028113640 con 0x7f602806b180 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f602dbc7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f60281134c0 con 0x7f6028109510 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f602dbc7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60281137c0 con 0x7f602806bb40 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36204/0 (socket says 172.21.15.3:36204) 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 -- 172.21.15.3:0/141316631 learned_addr learned my addr 172.21.15.3:0/141316631 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f6027fff700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6028109510 0x7f6028199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f6026ffd700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f602806bb40 0x7f602819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 -- 172.21.15.3:0/141316631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f602806bb40 msgr2=0x7f602819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f602806bb40 0x7f602819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 -- 172.21.15.3:0/141316631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6028109510 msgr2=0x7f6028199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6028109510 0x7f6028199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 -- 172.21.15.3:0/141316631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602819a0e0 con 0x7f602806b180 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f6026ffd700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f602806bb40 0x7f602819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.272+0000 7f60277fe700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602819f7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f601800d980 tx=0x7f601800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.273+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018017070 con 0x7f602806b180 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.273+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602819a3d0 con 0x7f602806b180 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.273+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6018005020 con 0x7f602806b180 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.273+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018009b50 con 0x7f602806b180 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.273+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60281a6690 con 0x7f602806b180 2024-05-08T05:51:23.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.274+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602806a190 con 0x7f602806b180 2024-05-08T05:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.276+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f60180051d0 con 0x7f602806b180 2024-05-08T05:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.277+0000 7f6024ff9700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60100b2910 0x7f60100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.277+0000 7f6026ffd700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60100b2910 0x7f60100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.277+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f60180e2850 con 0x7f602806b180 2024-05-08T05:51:23.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.277+0000 7f6026ffd700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60100b2910 0x7f60100b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f601c006010 tx=0x7f601c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:23.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.280+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6018020030 con 0x7f602806b180 2024-05-08T05:51:23.544 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.544+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f602802e510 con 0x7f602806b180 2024-05-08T05:51:23.545 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.544+0000 7f6024ff9700 1 -- 172.21.15.3:0/141316631 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7f60180054f0 con 0x7f602806b180 2024-05-08T05:51:23.545 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:23.545 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:51:23.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60100b2910 msgr2=0x7f60100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60100b2910 0x7f60100b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f601c006010 tx=0x7f601c00b540 comp rx=0 tx=0).stop 2024-05-08T05:51:23.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 msgr2=0x7f602819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:23.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602819f7c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f601800d980 tx=0x7f601800dc90 comp rx=0 tx=0).stop 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 shutdown_connections 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6028109510 0x7f6028199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f60100b2910 0x7f60100b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f602806b180 0x7f602819f7c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 --2- 172.21.15.3:0/141316631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f602806bb40 0x7f602819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 >> 172.21.15.3:0/141316631 conn(0x7f6028079350 msgr2=0x7f6028101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 shutdown_connections 2024-05-08T05:51:23.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:23.547+0000 7f602dbc7700 1 -- 172.21.15.3:0/141316631 wait complete. 2024-05-08T05:51:23.549 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:51:23.863 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:23 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/141316631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:51:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:23 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/141316631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:51:24.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:23 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/141316631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:51:24.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:51:24.445 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:24.892 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:24 smithi003 ceph-mon[30137]: pgmap v342: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-05-08T05:51:25.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:24 smithi089 ceph-mon[34495]: pgmap v342: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-05-08T05:51:25.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:24 smithi104 ceph-mon[34559]: pgmap v342: 129 pgs: 129 active+clean; 632 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-05-08T05:51:26.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 -- 172.21.15.3:0/2306826276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e341076c0 msgr2=0x7f0e3410f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/2306826276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e341076c0 0x7f0e3410f290 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00b0d0 tx=0x7f0e2c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 -- 172.21.15.3:0/2306826276 shutdown_connections 2024-05-08T05:51:26.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/2306826276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e341076c0 0x7f0e3410f290 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/2306826276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e34102630 0x7f0e34102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/2306826276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 0x7f0e34106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 -- 172.21.15.3:0/2306826276 >> 172.21.15.3:0/2306826276 conn(0x7f0e34079350 msgr2=0x7f0e34079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 -- 172.21.15.3:0/2306826276 shutdown_connections 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.424+0000 7f0e3b27f700 1 -- 172.21.15.3:0/2306826276 wait complete. 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 Processor -- start 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 -- start start 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 0x7f0e3419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 0x7f0e3419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e341076c0 0x7f0e341a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34112580 con 0x7f0e34102630 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34112400 con 0x7f0e34102fe0 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3b27f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34112700 con 0x7f0e341076c0 2024-05-08T05:51:26.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3901b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 0x7f0e3419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3901b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 0x7f0e3419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36222/0 (socket says 172.21.15.3:36222) 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3901b700 1 -- 172.21.15.3:0/262514853 learned_addr learned my addr 172.21.15.3:0/262514853 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3981c700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e341076c0 0x7f0e341a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.425+0000 7f0e3881a700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 0x7f0e3419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3901b700 1 -- 172.21.15.3:0/262514853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e341076c0 msgr2=0x7f0e341a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3901b700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e341076c0 0x7f0e341a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3901b700 1 -- 172.21.15.3:0/262514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 msgr2=0x7f0e3419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3901b700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 0x7f0e3419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3901b700 1 -- 172.21.15.3:0/262514853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e341a5fa0 con 0x7f0e34102630 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3881a700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 0x7f0e3419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3901b700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 0x7f0e3419a9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d980 tx=0x7f0e3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30017070 con 0x7f0e34102630 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e30005020 con 0x7f0e34102630 2024-05-08T05:51:26.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e341a6170 con 0x7f0e34102630 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30009b50 con 0x7f0e34102630 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.426+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e341a6630 con 0x7f0e34102630 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.427+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3406a190 con 0x7f0e34102630 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.429+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0e300051d0 con 0x7f0e34102630 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.429+0000 7f0e2a7fc700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0e200b2910 0x7f0e200b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.430+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f0e300e2a30 con 0x7f0e34102630 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.430+0000 7f0e3881a700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0e200b2910 0x7f0e200b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:26.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.430+0000 7f0e3881a700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0e200b2910 0x7f0e200b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005e10 tx=0x7f0e24005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:26.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.434+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e30020020 con 0x7f0e34102630 2024-05-08T05:51:26.661 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.661+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e3402e6d0 con 0x7f0e34102630 2024-05-08T05:51:26.662 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.663+0000 7f0e2a7fc700 1 -- 172.21.15.3:0/262514853 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+24572 (secure 0 0 0) 0x7f0e300aa290 con 0x7f0e34102630 2024-05-08T05:51:26.663 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:26.663 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":87,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:50:34.293465+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0e200b2910 msgr2=0x7f0e200b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0e200b2910 0x7f0e200b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0e24005e10 tx=0x7f0e24005d60 comp rx=0 tx=0).stop 2024-05-08T05:51:26.665 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 msgr2=0x7f0e3419a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 0x7f0e3419a9e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d980 tx=0x7f0e3000dc90 comp rx=0 tx=0).stop 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 shutdown_connections 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e341076c0 0x7f0e341a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0e200b2910 0x7f0e200b4dd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0e34102630 0x7f0e3419a9e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 --2- 172.21.15.3:0/262514853 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0e34102fe0 0x7f0e3419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 >> 172.21.15.3:0/262514853 conn(0x7f0e34079350 msgr2=0x7f0e34100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 shutdown_connections 2024-05-08T05:51:26.666 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:26.665+0000 7f0e3b27f700 1 -- 172.21.15.3:0/262514853 wait complete. 2024-05-08T05:51:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:26 smithi003 ceph-mon[30137]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-05-08T05:51:27.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:26 smithi089 ceph-mon[34495]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-05-08T05:51:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:26 smithi104 ceph-mon[34559]: pgmap v343: 129 pgs: 129 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-05-08T05:51:27.456 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:51:27.457 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:51:27.711 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:27.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:27 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/262514853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:27 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/262514853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:28.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:27 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/262514853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:28 smithi003 ceph-mon[30137]: pgmap v344: 129 pgs: 129 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-05-08T05:51:29.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:28 smithi089 ceph-mon[34495]: pgmap v344: 129 pgs: 129 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-05-08T05:51:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:28 smithi104 ceph-mon[34559]: pgmap v344: 129 pgs: 129 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 -- 172.21.15.3:0/3277057647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690108870 msgr2=0x7f16901103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 --2- 172.21.15.3:0/3277057647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690108870 0x7f16901103f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f168800b0d0 tx=0x7f168800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 -- 172.21.15.3:0/3277057647 shutdown_connections 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 --2- 172.21.15.3:0/3277057647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690108870 0x7f16901103f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 --2- 172.21.15.3:0/3277057647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690103780 0x7f1690103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 --2- 172.21.15.3:0/3277057647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690104130 0x7f1690108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 -- 172.21.15.3:0/3277057647 >> 172.21.15.3:0/3277057647 conn(0x7f169007a610 msgr2=0x7f169007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 -- 172.21.15.3:0/3277057647 shutdown_connections 2024-05-08T05:51:29.495 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.494+0000 7f16982a6700 1 -- 172.21.15.3:0/3277057647 wait complete. 2024-05-08T05:51:29.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.495+0000 7f16982a6700 1 Processor -- start 2024-05-08T05:51:29.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.495+0000 7f16982a6700 1 -- start start 2024-05-08T05:51:29.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16982a6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690103780 0x7f169019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16982a6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 0x7f169019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16982a6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690108870 0x7f1690199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.497 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16982a6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f16901136b0 con 0x7f1690104130 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16982a6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1690113530 con 0x7f1690108870 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16982a6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1690113830 con 0x7f1690103780 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 0x7f169019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 0x7f169019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36232/0 (socket says 172.21.15.3:36232) 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 -- 172.21.15.3:0/3094749587 learned_addr learned my addr 172.21.15.3:0/3094749587 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1696042700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690103780 0x7f169019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1696843700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690108870 0x7f1690199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 -- 172.21.15.3:0/3094749587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690103780 msgr2=0x7f169019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690103780 0x7f169019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 -- 172.21.15.3:0/3094749587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690108870 msgr2=0x7f1690199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690108870 0x7f1690199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.498 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 -- 172.21.15.3:0/3094749587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169019a090 con 0x7f1690104130 2024-05-08T05:51:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f1695841700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 0x7f169019fc60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f168000c920 tx=0x7f168000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.496+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680004a30 con 0x7f1690104130 2024-05-08T05:51:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.497+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169019a380 con 0x7f1690104130 2024-05-08T05:51:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.497+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16800054d0 con 0x7f1690104130 2024-05-08T05:51:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.497+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1680007bd0 con 0x7f1690104130 2024-05-08T05:51:29.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.497+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16901a65f0 con 0x7f1690104130 2024-05-08T05:51:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.498+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f169010ba40 con 0x7f1690104130 2024-05-08T05:51:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.499+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f1680004bd0 con 0x7f1690104130 2024-05-08T05:51:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.500+0000 7f16877fe700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f167c0b2910 0x7f167c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.500+0000 7f1696042700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f167c0b2910 0x7f167c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.500+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f16800e22a0 con 0x7f1690104130 2024-05-08T05:51:29.500 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.500+0000 7f1696042700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f167c0b2910 0x7f167c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f168c005950 tx=0x7f168c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:29.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.504+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1680014030 con 0x7f1690104130 2024-05-08T05:51:29.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.720+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16900638e0 con 0x7f167c0b2910 2024-05-08T05:51:29.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.730+0000 7f16877fe700 1 -- 172.21.15.3:0/3094749587 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340544 (secure 0 0 0) 0x7f16900638e0 con 0x7f167c0b2910 2024-05-08T05:51:29.735 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f167c0b2910 msgr2=0x7f167c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f167c0b2910 0x7f167c0b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f168c005950 tx=0x7f168c00a300 comp rx=0 tx=0).stop 2024-05-08T05:51:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 msgr2=0x7f169019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 0x7f169019fc60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f168000c920 tx=0x7f168000cc30 comp rx=0 tx=0).stop 2024-05-08T05:51:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 shutdown_connections 2024-05-08T05:51:29.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1690108870 0x7f1690199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1690103780 0x7f169019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f167c0b2910 0x7f167c0b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 --2- 172.21.15.3:0/3094749587 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1690104130 0x7f169019fc60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 >> 172.21.15.3:0/3094749587 conn(0x7f169007a610 msgr2=0x7f1690101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 shutdown_connections 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:29.733+0000 7f16982a6700 1 -- 172.21.15.3:0/3094749587 wait complete. 2024-05-08T05:51:29.739 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:51:30.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:30 smithi003 ceph-mon[30137]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.9 KiB/s wr, 5 op/s 2024-05-08T05:51:30.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:30 smithi003 ceph-mon[30137]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:51:30.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:30 smithi089 ceph-mon[34495]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.9 KiB/s wr, 5 op/s 2024-05-08T05:51:30.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:30 smithi089 ceph-mon[34495]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:51:30.516 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-08T05:51:29.236324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656527,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":147,"num_read_kb":163,"num_write":215,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360696,"kb_used_data":11528,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581128,"statfs":{"total":1151940427776,"available":1151571075072,"internally_reserved":0,"allocated":11804672,"data_stored":7883990,"data_compressed":424473,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36037,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":51,"num_write":29,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003033"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309094+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:50:34.309094+0000","last_peered":"2024-05-08T05:50:34.309094+0000","last_clean":"2024-05-08T05:50:34.309094+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:50:34.309094+0000","last_undegraded":"2024-05-08T05:50:34.309094+0000","last_fullsized":"2024-05-08T05:50:34.309094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301929+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:50:34.301929+0000","last_peered":"2024-05-08T05:50:34.301929+0000","last_clean":"2024-05-08T05:50:34.301929+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:50:34.301929+0000","last_undegraded":"2024-05-08T05:50:34.301929+0000","last_fullsized":"2024-05-08T05:50:34.301929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301712+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:50:34.301712+0000","last_peered":"2024-05-08T05:50:34.301712+0000","last_clean":"2024-05-08T05:50:34.301712+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:50:34.301712+0000","last_undegraded":"2024-05-08T05:50:34.301712+0000","last_fullsized":"2024-05-08T05:50:34.301712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299566+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:50:34.299566+0000","last_peered":"2024-05-08T05:50:34.299566+0000","last_clean":"2024-05-08T05:50:34.299566+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:50:34.299566+0000","last_undegraded":"2024-05-08T05:50:34.299566+0000","last_fullsized":"2024-05-08T05:50:34.299566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299857+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:50:34.299857+0000","last_peered":"2024-05-08T05:50:34.299857+0000","last_clean":"2024-05-08T05:50:34.299857+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:50:34.299857+0000","last_undegraded":"2024-05-08T05:50:34.299857+0000","last_fullsized":"2024-05-08T05:50:34.299857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302516+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:50:34.302516+0000","last_peered":"2024-05-08T05:50:34.302516+0000","last_clean":"2024-05-08T05:50:34.302516+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:50:34.302516+0000","last_undegraded":"2024-05-08T05:50:34.302516+0000","last_fullsized":"2024-05-08T05:50:34.302516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493816+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:50:34.493816+0000","last_peered":"2024-05-08T05:50:34.493816+0000","last_clean":"2024-05-08T05:50:34.493816+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:50:34.493816+0000","last_undegraded":"2024-05-08T05:50:34.493816+0000","last_fullsized":"2024-05-08T05:50:34.493816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371124+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:50:34.371124+0000","last_peered":"2024-05-08T05:50:34.371124+0000","last_clean":"2024-05-08T05:50:34.371124+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:50:34.371124+0000","last_undegraded":"2024-05-08T05:50:34.371124+0000","last_fullsized":"2024-05-08T05:50:34.371124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306446+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:50:34.306446+0000","last_peered":"2024-05-08T05:50:34.306446+0000","last_clean":"2024-05-08T05:50:34.306446+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:50:34.306446+0000","last_undegraded":"2024-05-08T05:50:34.306446+0000","last_fullsized":"2024-05-08T05:50:34.306446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302327+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:50:34.302327+0000","last_peered":"2024-05-08T05:50:34.302327+0000","last_clean":"2024-05-08T05:50:34.302327+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:50:34.302327+0000","last_undegraded":"2024-05-08T05:50:34.302327+0000","last_fullsized":"2024-05-08T05:50:34.302327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494192+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:50:34.494192+0000","last_peered":"2024-05-08T05:50:34.494192+0000","last_clean":"2024-05-08T05:50:34.494192+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:50:34.494192+0000","last_undegraded":"2024-05-08T05:50:34.494192+0000","last_fullsized":"2024-05-08T05:50:34.494192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299489+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:50:34.299489+0000","last_peered":"2024-05-08T05:50:34.299489+0000","last_clean":"2024-05-08T05:50:34.299489+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:50:34.299489+0000","last_undegraded":"2024-05-08T05:50:34.299489+0000","last_fullsized":"2024-05-08T05:50:34.299489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382579+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:50:34.382579+0000","last_peered":"2024-05-08T05:50:34.382579+0000","last_clean":"2024-05-08T05:50:34.382579+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:50:34.382579+0000","last_undegraded":"2024-05-08T05:50:34.382579+0000","last_fullsized":"2024-05-08T05:50:34.382579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301354+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:50:34.301354+0000","last_peered":"2024-05-08T05:50:34.301354+0000","last_clean":"2024-05-08T05:50:34.301354+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:50:34.301354+0000","last_undegraded":"2024-05-08T05:50:34.301354+0000","last_fullsized":"2024-05-08T05:50:34.301354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301406+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:50:34.301406+0000","last_peered":"2024-05-08T05:50:34.301406+0000","last_clean":"2024-05-08T05:50:34.301406+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:50:34.301406+0000","last_undegraded":"2024-05-08T05:50:34.301406+0000","last_fullsized":"2024-05-08T05:50:34.301406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305522+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:50:34.305522+0000","last_peered":"2024-05-08T05:50:34.305522+0000","last_clean":"2024-05-08T05:50:34.305522+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:50:34.305522+0000","last_undegraded":"2024-05-08T05:50:34.305522+0000","last_fullsized":"2024-05-08T05:50:34.305522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308757+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:50:34.308757+0000","last_peered":"2024-05-08T05:50:34.308757+0000","last_clean":"2024-05-08T05:50:34.308757+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:50:34.308757+0000","last_undegraded":"2024-05-08T05:50:34.308757+0000","last_fullsized":"2024-05-08T05:50:34.308757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305027+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:50:34.305027+0000","last_peered":"2024-05-08T05:50:34.305027+0000","last_clean":"2024-05-08T05:50:34.305027+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:50:34.305027+0000","last_undegraded":"2024-05-08T05:50:34.305027+0000","last_fullsized":"2024-05-08T05:50:34.305027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304350+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:50:34.304350+0000","last_peered":"2024-05-08T05:50:34.304350+0000","last_clean":"2024-05-08T05:50:34.304350+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:50:34.304350+0000","last_undegraded":"2024-05-08T05:50:34.304350+0000","last_fullsized":"2024-05-08T05:50:34.304350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302774+0000","last_change":"2024-05-08T05:50:12.705411+0000","last_active":"2024-05-08T05:50:34.302774+0000","last_peered":"2024-05-08T05:50:34.302774+0000","last_clean":"2024-05-08T05:50:34.302774+0000","last_became_active":"2024-05-08T05:50:12.704966+0000","last_became_peered":"2024-05-08T05:50:12.704966+0000","last_unstale":"2024-05-08T05:50:34.302774+0000","last_undegraded":"2024-05-08T05:50:34.302774+0000","last_fullsized":"2024-05-08T05:50:34.302774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:35.286238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305173+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:50:34.305173+0000","last_peered":"2024-05-08T05:50:34.305173+0000","last_clean":"2024-05-08T05:50:34.305173+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:50:34.305173+0000","last_undegraded":"2024-05-08T05:50:34.305173+0000","last_fullsized":"2024-05-08T05:50:34.305173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371532+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:50:34.371532+0000","last_peered":"2024-05-08T05:50:34.371532+0000","last_clean":"2024-05-08T05:50:34.371532+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:50:34.371532+0000","last_undegraded":"2024-05-08T05:50:34.371532+0000","last_fullsized":"2024-05-08T05:50:34.371532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299072+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:50:34.299072+0000","last_peered":"2024-05-08T05:50:34.299072+0000","last_clean":"2024-05-08T05:50:34.299072+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:50:34.299072+0000","last_undegraded":"2024-05-08T05:50:34.299072+0000","last_fullsized":"2024-05-08T05:50:34.299072+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371651+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:50:34.371651+0000","last_peered":"2024-05-08T05:50:34.371651+0000","last_clean":"2024-05-08T05:50:34.371651+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:50:34.371651+0000","last_undegraded":"2024-05-08T05:50:34.371651+0000","last_fullsized":"2024-05-08T05:50:34.371651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302007+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:50:34.302007+0000","last_peered":"2024-05-08T05:50:34.302007+0000","last_clean":"2024-05-08T05:50:34.302007+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:50:34.302007+0000","last_undegraded":"2024-05-08T05:50:34.302007+0000","last_fullsized":"2024-05-08T05:50:34.302007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299245+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:50:34.299245+0000","last_peered":"2024-05-08T05:50:34.299245+0000","last_clean":"2024-05-08T05:50:34.299245+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:50:34.299245+0000","last_undegraded":"2024-05-08T05:50:34.299245+0000","last_fullsized":"2024-05-08T05:50:34.299245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300482+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:50:34.300482+0000","last_peered":"2024-05-08T05:50:34.300482+0000","last_clean":"2024-05-08T05:50:34.300482+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:50:34.300482+0000","last_undegraded":"2024-05-08T05:50:34.300482+0000","last_fullsized":"2024-05-08T05:50:34.300482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306136+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:50:34.306136+0000","last_peered":"2024-05-08T05:50:34.306136+0000","last_clean":"2024-05-08T05:50:34.306136+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:50:34.306136+0000","last_undegraded":"2024-05-08T05:50:34.306136+0000","last_fullsized":"2024-05-08T05:50:34.306136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299191+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:50:34.299191+0000","last_peered":"2024-05-08T05:50:34.299191+0000","last_clean":"2024-05-08T05:50:34.299191+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:50:34.299191+0000","last_undegraded":"2024-05-08T05:50:34.299191+0000","last_fullsized":"2024-05-08T05:50:34.299191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.314163+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:50:34.314163+0000","last_peered":"2024-05-08T05:50:34.314163+0000","last_clean":"2024-05-08T05:50:34.314163+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:50:34.314163+0000","last_undegraded":"2024-05-08T05:50:34.314163+0000","last_fullsized":"2024-05-08T05:50:34.314163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.383855+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:50:34.383855+0000","last_peered":"2024-05-08T05:50:34.383855+0000","last_clean":"2024-05-08T05:50:34.383855+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:50:34.383855+0000","last_undegraded":"2024-05-08T05:50:34.383855+0000","last_fullsized":"2024-05-08T05:50:34.383855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304090+0000","last_change":"2024-05-08T05:50:12.707786+0000","last_active":"2024-05-08T05:50:34.304090+0000","last_peered":"2024-05-08T05:50:34.304090+0000","last_clean":"2024-05-08T05:50:34.304090+0000","last_became_active":"2024-05-08T05:50:12.707349+0000","last_became_peered":"2024-05-08T05:50:12.707349+0000","last_unstale":"2024-05-08T05:50:34.304090+0000","last_undegraded":"2024-05-08T05:50:34.304090+0000","last_fullsized":"2024-05-08T05:50:34.304090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:52.337370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304616+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:50:34.304616+0000","last_peered":"2024-05-08T05:50:34.304616+0000","last_clean":"2024-05-08T05:50:34.304616+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:50:34.304616+0000","last_undegraded":"2024-05-08T05:50:34.304616+0000","last_fullsized":"2024-05-08T05:50:34.304616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299828+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:50:34.299828+0000","last_peered":"2024-05-08T05:50:34.299828+0000","last_clean":"2024-05-08T05:50:34.299828+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:50:34.299828+0000","last_undegraded":"2024-05-08T05:50:34.299828+0000","last_fullsized":"2024-05-08T05:50:34.299828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303784+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:50:34.303784+0000","last_peered":"2024-05-08T05:50:34.303784+0000","last_clean":"2024-05-08T05:50:34.303784+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:50:34.303784+0000","last_undegraded":"2024-05-08T05:50:34.303784+0000","last_fullsized":"2024-05-08T05:50:34.303784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302445+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:50:34.302445+0000","last_peered":"2024-05-08T05:50:34.302445+0000","last_clean":"2024-05-08T05:50:34.302445+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:50:34.302445+0000","last_undegraded":"2024-05-08T05:50:34.302445+0000","last_fullsized":"2024-05-08T05:50:34.302445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.373044+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:50:34.373044+0000","last_peered":"2024-05-08T05:50:34.373044+0000","last_clean":"2024-05-08T05:50:34.373044+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:50:34.373044+0000","last_undegraded":"2024-05-08T05:50:34.373044+0000","last_fullsized":"2024-05-08T05:50:34.373044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308643+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:50:34.308643+0000","last_peered":"2024-05-08T05:50:34.308643+0000","last_clean":"2024-05-08T05:50:34.308643+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:50:34.308643+0000","last_undegraded":"2024-05-08T05:50:34.308643+0000","last_fullsized":"2024-05-08T05:50:34.308643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382236+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:50:34.382236+0000","last_peered":"2024-05-08T05:50:34.382236+0000","last_clean":"2024-05-08T05:50:34.382236+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:50:34.382236+0000","last_undegraded":"2024-05-08T05:50:34.382236+0000","last_fullsized":"2024-05-08T05:50:34.382236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371244+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:50:34.371244+0000","last_peered":"2024-05-08T05:50:34.371244+0000","last_clean":"2024-05-08T05:50:34.371244+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:50:34.371244+0000","last_undegraded":"2024-05-08T05:50:34.371244+0000","last_fullsized":"2024-05-08T05:50:34.371244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301740+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:50:34.301740+0000","last_peered":"2024-05-08T05:50:34.301740+0000","last_clean":"2024-05-08T05:50:34.301740+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:50:34.301740+0000","last_undegraded":"2024-05-08T05:50:34.301740+0000","last_fullsized":"2024-05-08T05:50:34.301740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304326+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:50:34.304326+0000","last_peered":"2024-05-08T05:50:34.304326+0000","last_clean":"2024-05-08T05:50:34.304326+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:50:34.304326+0000","last_undegraded":"2024-05-08T05:50:34.304326+0000","last_fullsized":"2024-05-08T05:50:34.304326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304401+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:50:34.304401+0000","last_peered":"2024-05-08T05:50:34.304401+0000","last_clean":"2024-05-08T05:50:34.304401+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:50:34.304401+0000","last_undegraded":"2024-05-08T05:50:34.304401+0000","last_fullsized":"2024-05-08T05:50:34.304401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371810+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:50:34.371810+0000","last_peered":"2024-05-08T05:50:34.371810+0000","last_clean":"2024-05-08T05:50:34.371810+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:50:34.371810+0000","last_undegraded":"2024-05-08T05:50:34.371810+0000","last_fullsized":"2024-05-08T05:50:34.371810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302379+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:50:34.302379+0000","last_peered":"2024-05-08T05:50:34.302379+0000","last_clean":"2024-05-08T05:50:34.302379+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:50:34.302379+0000","last_undegraded":"2024-05-08T05:50:34.302379+0000","last_fullsized":"2024-05-08T05:50:34.302379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306647+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:50:34.306647+0000","last_peered":"2024-05-08T05:50:34.306647+0000","last_clean":"2024-05-08T05:50:34.306647+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:50:34.306647+0000","last_undegraded":"2024-05-08T05:50:34.306647+0000","last_fullsized":"2024-05-08T05:50:34.306647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303999+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:50:34.303999+0000","last_peered":"2024-05-08T05:50:34.303999+0000","last_clean":"2024-05-08T05:50:34.303999+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:50:34.303999+0000","last_undegraded":"2024-05-08T05:50:34.303999+0000","last_fullsized":"2024-05-08T05:50:34.303999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494225+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:50:34.494225+0000","last_peered":"2024-05-08T05:50:34.494225+0000","last_clean":"2024-05-08T05:50:34.494225+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:50:34.494225+0000","last_undegraded":"2024-05-08T05:50:34.494225+0000","last_fullsized":"2024-05-08T05:50:34.494225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301971+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:50:34.301971+0000","last_peered":"2024-05-08T05:50:34.301971+0000","last_clean":"2024-05-08T05:50:34.301971+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:50:34.301971+0000","last_undegraded":"2024-05-08T05:50:34.301971+0000","last_fullsized":"2024-05-08T05:50:34.301971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299540+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:50:34.299540+0000","last_peered":"2024-05-08T05:50:34.299540+0000","last_clean":"2024-05-08T05:50:34.299540+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:50:34.299540+0000","last_undegraded":"2024-05-08T05:50:34.299540+0000","last_fullsized":"2024-05-08T05:50:34.299540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304053+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:50:34.304053+0000","last_peered":"2024-05-08T05:50:34.304053+0000","last_clean":"2024-05-08T05:50:34.304053+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:50:34.304053+0000","last_undegraded":"2024-05-08T05:50:34.304053+0000","last_fullsized":"2024-05-08T05:50:34.304053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305890+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:50:34.305890+0000","last_peered":"2024-05-08T05:50:34.305890+0000","last_clean":"2024-05-08T05:50:34.305890+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:50:34.305890+0000","last_undegraded":"2024-05-08T05:50:34.305890+0000","last_fullsized":"2024-05-08T05:50:34.305890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308283+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:50:34.308283+0000","last_peered":"2024-05-08T05:50:34.308283+0000","last_clean":"2024-05-08T05:50:34.308283+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:50:34.308283+0000","last_undegraded":"2024-05-08T05:50:34.308283+0000","last_fullsized":"2024-05-08T05:50:34.308283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:51:28.493623+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:51:28.493623+0000","last_peered":"2024-05-08T05:51:28.493623+0000","last_clean":"2024-05-08T05:51:28.493623+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:51:28.493623+0000","last_undegraded":"2024-05-08T05:51:28.493623+0000","last_fullsized":"2024-05-08T05:51:28.493623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65127,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":60,"num_read_kb":88,"num_write":33,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300373+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:50:34.300373+0000","last_peered":"2024-05-08T05:50:34.300373+0000","last_clean":"2024-05-08T05:50:34.300373+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:50:34.300373+0000","last_undegraded":"2024-05-08T05:50:34.300373+0000","last_fullsized":"2024-05-08T05:50:34.300373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305497+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:50:34.305497+0000","last_peered":"2024-05-08T05:50:34.305497+0000","last_clean":"2024-05-08T05:50:34.305497+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:50:34.305497+0000","last_undegraded":"2024-05-08T05:50:34.305497+0000","last_fullsized":"2024-05-08T05:50:34.305497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303591+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:50:34.303591+0000","last_peered":"2024-05-08T05:50:34.303591+0000","last_clean":"2024-05-08T05:50:34.303591+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:50:34.303591+0000","last_undegraded":"2024-05-08T05:50:34.303591+0000","last_fullsized":"2024-05-08T05:50:34.303591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382278+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:50:34.382278+0000","last_peered":"2024-05-08T05:50:34.382278+0000","last_clean":"2024-05-08T05:50:34.382278+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:50:34.382278+0000","last_undegraded":"2024-05-08T05:50:34.382278+0000","last_fullsized":"2024-05-08T05:50:34.382278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309374+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:50:34.309374+0000","last_peered":"2024-05-08T05:50:34.309374+0000","last_clean":"2024-05-08T05:50:34.309374+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:50:34.309374+0000","last_undegraded":"2024-05-08T05:50:34.309374+0000","last_fullsized":"2024-05-08T05:50:34.309374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305212+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:50:34.305212+0000","last_peered":"2024-05-08T05:50:34.305212+0000","last_clean":"2024-05-08T05:50:34.305212+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:50:34.305212+0000","last_undegraded":"2024-05-08T05:50:34.305212+0000","last_fullsized":"2024-05-08T05:50:34.305212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493970+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:50:34.493970+0000","last_peered":"2024-05-08T05:50:34.493970+0000","last_clean":"2024-05-08T05:50:34.493970+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:50:34.493970+0000","last_undegraded":"2024-05-08T05:50:34.493970+0000","last_fullsized":"2024-05-08T05:50:34.493970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304887+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:50:34.304887+0000","last_peered":"2024-05-08T05:50:34.304887+0000","last_clean":"2024-05-08T05:50:34.304887+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:50:34.304887+0000","last_undegraded":"2024-05-08T05:50:34.304887+0000","last_fullsized":"2024-05-08T05:50:34.304887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299136+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:50:34.299136+0000","last_peered":"2024-05-08T05:50:34.299136+0000","last_clean":"2024-05-08T05:50:34.299136+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:50:34.299136+0000","last_undegraded":"2024-05-08T05:50:34.299136+0000","last_fullsized":"2024-05-08T05:50:34.299136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493834+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:50:34.493834+0000","last_peered":"2024-05-08T05:50:34.493834+0000","last_clean":"2024-05-08T05:50:34.493834+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:50:34.493834+0000","last_undegraded":"2024-05-08T05:50:34.493834+0000","last_fullsized":"2024-05-08T05:50:34.493834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382601+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:50:34.382601+0000","last_peered":"2024-05-08T05:50:34.382601+0000","last_clean":"2024-05-08T05:50:34.382601+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:50:34.382601+0000","last_undegraded":"2024-05-08T05:50:34.382601+0000","last_fullsized":"2024-05-08T05:50:34.382601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.315867+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:50:34.315867+0000","last_peered":"2024-05-08T05:50:34.315867+0000","last_clean":"2024-05-08T05:50:34.315867+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:50:34.315867+0000","last_undegraded":"2024-05-08T05:50:34.315867+0000","last_fullsized":"2024-05-08T05:50:34.315867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306536+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:50:34.306536+0000","last_peered":"2024-05-08T05:50:34.306536+0000","last_clean":"2024-05-08T05:50:34.306536+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:50:34.306536+0000","last_undegraded":"2024-05-08T05:50:34.306536+0000","last_fullsized":"2024-05-08T05:50:34.306536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306413+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:50:34.306413+0000","last_peered":"2024-05-08T05:50:34.306413+0000","last_clean":"2024-05-08T05:50:34.306413+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:50:34.306413+0000","last_undegraded":"2024-05-08T05:50:34.306413+0000","last_fullsized":"2024-05-08T05:50:34.306413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300797+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:50:34.300797+0000","last_peered":"2024-05-08T05:50:34.300797+0000","last_clean":"2024-05-08T05:50:34.300797+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:50:34.300797+0000","last_undegraded":"2024-05-08T05:50:34.300797+0000","last_fullsized":"2024-05-08T05:50:34.300797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.310426+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:50:34.310426+0000","last_peered":"2024-05-08T05:50:34.310426+0000","last_clean":"2024-05-08T05:50:34.310426+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:50:34.310426+0000","last_undegraded":"2024-05-08T05:50:34.310426+0000","last_fullsized":"2024-05-08T05:50:34.310426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309753+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:50:34.309753+0000","last_peered":"2024-05-08T05:50:34.309753+0000","last_clean":"2024-05-08T05:50:34.309753+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:50:34.309753+0000","last_undegraded":"2024-05-08T05:50:34.309753+0000","last_fullsized":"2024-05-08T05:50:34.309753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.312120+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:50:34.312120+0000","last_peered":"2024-05-08T05:50:34.312120+0000","last_clean":"2024-05-08T05:50:34.312120+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:50:34.312120+0000","last_undegraded":"2024-05-08T05:50:34.312120+0000","last_fullsized":"2024-05-08T05:50:34.312120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301983+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:50:34.301983+0000","last_peered":"2024-05-08T05:50:34.301983+0000","last_clean":"2024-05-08T05:50:34.301983+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:50:34.301983+0000","last_undegraded":"2024-05-08T05:50:34.301983+0000","last_fullsized":"2024-05-08T05:50:34.301983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382552+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:50:34.382552+0000","last_peered":"2024-05-08T05:50:34.382552+0000","last_clean":"2024-05-08T05:50:34.382552+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:50:34.382552+0000","last_undegraded":"2024-05-08T05:50:34.382552+0000","last_fullsized":"2024-05-08T05:50:34.382552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493846+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:50:34.493846+0000","last_peered":"2024-05-08T05:50:34.493846+0000","last_clean":"2024-05-08T05:50:34.493846+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:50:34.493846+0000","last_undegraded":"2024-05-08T05:50:34.493846+0000","last_fullsized":"2024-05-08T05:50:34.493846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306858+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:50:34.306858+0000","last_peered":"2024-05-08T05:50:34.306858+0000","last_clean":"2024-05-08T05:50:34.306858+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:50:34.306858+0000","last_undegraded":"2024-05-08T05:50:34.306858+0000","last_fullsized":"2024-05-08T05:50:34.306858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382650+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:50:34.382650+0000","last_peered":"2024-05-08T05:50:34.382650+0000","last_clean":"2024-05-08T05:50:34.382650+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:50:34.382650+0000","last_undegraded":"2024-05-08T05:50:34.382650+0000","last_fullsized":"2024-05-08T05:50:34.382650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305055+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:50:34.305055+0000","last_peered":"2024-05-08T05:50:34.305055+0000","last_clean":"2024-05-08T05:50:34.305055+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:50:34.305055+0000","last_undegraded":"2024-05-08T05:50:34.305055+0000","last_fullsized":"2024-05-08T05:50:34.305055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382134+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:50:34.382134+0000","last_peered":"2024-05-08T05:50:34.382134+0000","last_clean":"2024-05-08T05:50:34.382134+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:50:34.382134+0000","last_undegraded":"2024-05-08T05:50:34.382134+0000","last_fullsized":"2024-05-08T05:50:34.382134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494165+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:50:34.494165+0000","last_peered":"2024-05-08T05:50:34.494165+0000","last_clean":"2024-05-08T05:50:34.494165+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:50:34.494165+0000","last_undegraded":"2024-05-08T05:50:34.494165+0000","last_fullsized":"2024-05-08T05:50:34.494165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371490+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:50:34.371490+0000","last_peered":"2024-05-08T05:50:34.371490+0000","last_clean":"2024-05-08T05:50:34.371490+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:50:34.371490+0000","last_undegraded":"2024-05-08T05:50:34.371490+0000","last_fullsized":"2024-05-08T05:50:34.371490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299546+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:50:34.299546+0000","last_peered":"2024-05-08T05:50:34.299546+0000","last_clean":"2024-05-08T05:50:34.299546+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:50:34.299546+0000","last_undegraded":"2024-05-08T05:50:34.299546+0000","last_fullsized":"2024-05-08T05:50:34.299546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493747+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:50:34.493747+0000","last_peered":"2024-05-08T05:50:34.493747+0000","last_clean":"2024-05-08T05:50:34.493747+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:50:34.493747+0000","last_undegraded":"2024-05-08T05:50:34.493747+0000","last_fullsized":"2024-05-08T05:50:34.493747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304375+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:50:34.304375+0000","last_peered":"2024-05-08T05:50:34.304375+0000","last_clean":"2024-05-08T05:50:34.304375+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:50:34.304375+0000","last_undegraded":"2024-05-08T05:50:34.304375+0000","last_fullsized":"2024-05-08T05:50:34.304375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382105+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:50:34.382105+0000","last_peered":"2024-05-08T05:50:34.382105+0000","last_clean":"2024-05-08T05:50:34.382105+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:50:34.382105+0000","last_undegraded":"2024-05-08T05:50:34.382105+0000","last_fullsized":"2024-05-08T05:50:34.382105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493827+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:50:34.493827+0000","last_peered":"2024-05-08T05:50:34.493827+0000","last_clean":"2024-05-08T05:50:34.493827+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:50:34.493827+0000","last_undegraded":"2024-05-08T05:50:34.493827+0000","last_fullsized":"2024-05-08T05:50:34.493827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307301+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:50:34.307301+0000","last_peered":"2024-05-08T05:50:34.307301+0000","last_clean":"2024-05-08T05:50:34.307301+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:50:34.307301+0000","last_undegraded":"2024-05-08T05:50:34.307301+0000","last_fullsized":"2024-05-08T05:50:34.307301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300158+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:50:34.300158+0000","last_peered":"2024-05-08T05:50:34.300158+0000","last_clean":"2024-05-08T05:50:34.300158+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:50:34.300158+0000","last_undegraded":"2024-05-08T05:50:34.300158+0000","last_fullsized":"2024-05-08T05:50:34.300158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382811+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:50:34.382811+0000","last_peered":"2024-05-08T05:50:34.382811+0000","last_clean":"2024-05-08T05:50:34.382811+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:50:34.382811+0000","last_undegraded":"2024-05-08T05:50:34.382811+0000","last_fullsized":"2024-05-08T05:50:34.382811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371130+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:50:34.371130+0000","last_peered":"2024-05-08T05:50:34.371130+0000","last_clean":"2024-05-08T05:50:34.371130+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:50:34.371130+0000","last_undegraded":"2024-05-08T05:50:34.371130+0000","last_fullsized":"2024-05-08T05:50:34.371130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:51:22.655968+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:51:22.655968+0000","last_peered":"2024-05-08T05:51:22.655968+0000","last_clean":"2024-05-08T05:51:22.655968+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:51:22.655968+0000","last_undegraded":"2024-05-08T05:51:22.655968+0000","last_fullsized":"2024-05-08T05:51:22.655968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371519+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:50:34.371519+0000","last_peered":"2024-05-08T05:50:34.371519+0000","last_clean":"2024-05-08T05:50:34.371519+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:50:34.371519+0000","last_undegraded":"2024-05-08T05:50:34.371519+0000","last_fullsized":"2024-05-08T05:50:34.371519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302619+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:50:34.302619+0000","last_peered":"2024-05-08T05:50:34.302619+0000","last_clean":"2024-05-08T05:50:34.302619+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:50:34.302619+0000","last_undegraded":"2024-05-08T05:50:34.302619+0000","last_fullsized":"2024-05-08T05:50:34.302619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300507+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:50:34.300507+0000","last_peered":"2024-05-08T05:50:34.300507+0000","last_clean":"2024-05-08T05:50:34.300507+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:50:34.300507+0000","last_undegraded":"2024-05-08T05:50:34.300507+0000","last_fullsized":"2024-05-08T05:50:34.300507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302069+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:50:34.302069+0000","last_peered":"2024-05-08T05:50:34.302069+0000","last_clean":"2024-05-08T05:50:34.302069+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:50:34.302069+0000","last_undegraded":"2024-05-08T05:50:34.302069+0000","last_fullsized":"2024-05-08T05:50:34.302069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299472+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:50:34.299472+0000","last_peered":"2024-05-08T05:50:34.299472+0000","last_clean":"2024-05-08T05:50:34.299472+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:50:34.299472+0000","last_undegraded":"2024-05-08T05:50:34.299472+0000","last_fullsized":"2024-05-08T05:50:34.299472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305592+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:50:34.305592+0000","last_peered":"2024-05-08T05:50:34.305592+0000","last_clean":"2024-05-08T05:50:34.305592+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:50:34.305592+0000","last_undegraded":"2024-05-08T05:50:34.305592+0000","last_fullsized":"2024-05-08T05:50:34.305592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299749+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:50:34.299749+0000","last_peered":"2024-05-08T05:50:34.299749+0000","last_clean":"2024-05-08T05:50:34.299749+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:50:34.299749+0000","last_undegraded":"2024-05-08T05:50:34.299749+0000","last_fullsized":"2024-05-08T05:50:34.299749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304873+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:50:34.304873+0000","last_peered":"2024-05-08T05:50:34.304873+0000","last_clean":"2024-05-08T05:50:34.304873+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:50:34.304873+0000","last_undegraded":"2024-05-08T05:50:34.304873+0000","last_fullsized":"2024-05-08T05:50:34.304873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371403+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:50:34.371403+0000","last_peered":"2024-05-08T05:50:34.371403+0000","last_clean":"2024-05-08T05:50:34.371403+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:50:34.371403+0000","last_undegraded":"2024-05-08T05:50:34.371403+0000","last_fullsized":"2024-05-08T05:50:34.371403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307337+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:50:34.307337+0000","last_peered":"2024-05-08T05:50:34.307337+0000","last_clean":"2024-05-08T05:50:34.307337+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:50:34.307337+0000","last_undegraded":"2024-05-08T05:50:34.307337+0000","last_fullsized":"2024-05-08T05:50:34.307337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299422+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:50:34.299422+0000","last_peered":"2024-05-08T05:50:34.299422+0000","last_clean":"2024-05-08T05:50:34.299422+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:50:34.299422+0000","last_undegraded":"2024-05-08T05:50:34.299422+0000","last_fullsized":"2024-05-08T05:50:34.299422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303161+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:50:34.303161+0000","last_peered":"2024-05-08T05:50:34.303161+0000","last_clean":"2024-05-08T05:50:34.303161+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:50:34.303161+0000","last_undegraded":"2024-05-08T05:50:34.303161+0000","last_fullsized":"2024-05-08T05:50:34.303161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371523+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:50:34.371523+0000","last_peered":"2024-05-08T05:50:34.371523+0000","last_clean":"2024-05-08T05:50:34.371523+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:50:34.371523+0000","last_undegraded":"2024-05-08T05:50:34.371523+0000","last_fullsized":"2024-05-08T05:50:34.371523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299549+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:50:34.299549+0000","last_peered":"2024-05-08T05:50:34.299549+0000","last_clean":"2024-05-08T05:50:34.299549+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:50:34.299549+0000","last_undegraded":"2024-05-08T05:50:34.299549+0000","last_fullsized":"2024-05-08T05:50:34.299549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299998+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:50:34.299998+0000","last_peered":"2024-05-08T05:50:34.299998+0000","last_clean":"2024-05-08T05:50:34.299998+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:50:34.299998+0000","last_undegraded":"2024-05-08T05:50:34.299998+0000","last_fullsized":"2024-05-08T05:50:34.299998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494039+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:50:34.494039+0000","last_peered":"2024-05-08T05:50:34.494039+0000","last_clean":"2024-05-08T05:50:34.494039+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:50:34.494039+0000","last_undegraded":"2024-05-08T05:50:34.494039+0000","last_fullsized":"2024-05-08T05:50:34.494039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305715+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:50:34.305715+0000","last_peered":"2024-05-08T05:50:34.305715+0000","last_clean":"2024-05-08T05:50:34.305715+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:50:34.305715+0000","last_undegraded":"2024-05-08T05:50:34.305715+0000","last_fullsized":"2024-05-08T05:50:34.305715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304866+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:50:34.304866+0000","last_peered":"2024-05-08T05:50:34.304866+0000","last_clean":"2024-05-08T05:50:34.304866+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:50:34.304866+0000","last_undegraded":"2024-05-08T05:50:34.304866+0000","last_fullsized":"2024-05-08T05:50:34.304866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494288+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:50:34.494288+0000","last_peered":"2024-05-08T05:50:34.494288+0000","last_clean":"2024-05-08T05:50:34.494288+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:50:34.494288+0000","last_undegraded":"2024-05-08T05:50:34.494288+0000","last_fullsized":"2024-05-08T05:50:34.494288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301345+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:50:34.301345+0000","last_peered":"2024-05-08T05:50:34.301345+0000","last_clean":"2024-05-08T05:50:34.301345+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:50:34.301345+0000","last_undegraded":"2024-05-08T05:50:34.301345+0000","last_fullsized":"2024-05-08T05:50:34.301345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304249+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:50:34.304249+0000","last_peered":"2024-05-08T05:50:34.304249+0000","last_clean":"2024-05-08T05:50:34.304249+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:50:34.304249+0000","last_undegraded":"2024-05-08T05:50:34.304249+0000","last_fullsized":"2024-05-08T05:50:34.304249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493978+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:50:34.493978+0000","last_peered":"2024-05-08T05:50:34.493978+0000","last_clean":"2024-05-08T05:50:34.493978+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:50:34.493978+0000","last_undegraded":"2024-05-08T05:50:34.493978+0000","last_fullsized":"2024-05-08T05:50:34.493978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304117+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:50:34.304117+0000","last_peered":"2024-05-08T05:50:34.304117+0000","last_clean":"2024-05-08T05:50:34.304117+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:50:34.304117+0000","last_undegraded":"2024-05-08T05:50:34.304117+0000","last_fullsized":"2024-05-08T05:50:34.304117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382642+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:50:34.382642+0000","last_peered":"2024-05-08T05:50:34.382642+0000","last_clean":"2024-05-08T05:50:34.382642+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:50:34.382642+0000","last_undegraded":"2024-05-08T05:50:34.382642+0000","last_fullsized":"2024-05-08T05:50:34.382642+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494351+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:50:34.494351+0000","last_peered":"2024-05-08T05:50:34.494351+0000","last_clean":"2024-05-08T05:50:34.494351+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:50:34.494351+0000","last_undegraded":"2024-05-08T05:50:34.494351+0000","last_fullsized":"2024-05-08T05:50:34.494351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307427+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:50:34.307427+0000","last_peered":"2024-05-08T05:50:34.307427+0000","last_clean":"2024-05-08T05:50:34.307427+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:50:34.307427+0000","last_undegraded":"2024-05-08T05:50:34.307427+0000","last_fullsized":"2024-05-08T05:50:34.307427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303042+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:50:34.303042+0000","last_peered":"2024-05-08T05:50:34.303042+0000","last_clean":"2024-05-08T05:50:34.303042+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:50:34.303042+0000","last_undegraded":"2024-05-08T05:50:34.303042+0000","last_fullsized":"2024-05-08T05:50:34.303042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303033+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:50:34.303033+0000","last_peered":"2024-05-08T05:50:34.303033+0000","last_clean":"2024-05-08T05:50:34.303033+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:50:34.303033+0000","last_undegraded":"2024-05-08T05:50:34.303033+0000","last_fullsized":"2024-05-08T05:50:34.303033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301837+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:50:34.301837+0000","last_peered":"2024-05-08T05:50:34.301837+0000","last_clean":"2024-05-08T05:50:34.301837+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:50:34.301837+0000","last_undegraded":"2024-05-08T05:50:34.301837+0000","last_fullsized":"2024-05-08T05:50:34.301837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382697+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:50:34.382697+0000","last_peered":"2024-05-08T05:50:34.382697+0000","last_clean":"2024-05-08T05:50:34.382697+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:50:34.382697+0000","last_undegraded":"2024-05-08T05:50:34.382697+0000","last_fullsized":"2024-05-08T05:50:34.382697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308345+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:50:34.308345+0000","last_peered":"2024-05-08T05:50:34.308345+0000","last_clean":"2024-05-08T05:50:34.308345+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:50:34.308345+0000","last_undegraded":"2024-05-08T05:50:34.308345+0000","last_fullsized":"2024-05-08T05:50:34.308345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304732+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:50:34.304732+0000","last_peered":"2024-05-08T05:50:34.304732+0000","last_clean":"2024-05-08T05:50:34.304732+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:50:34.304732+0000","last_undegraded":"2024-05-08T05:50:34.304732+0000","last_fullsized":"2024-05-08T05:50:34.304732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371721+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:50:34.371721+0000","last_peered":"2024-05-08T05:50:34.371721+0000","last_clean":"2024-05-08T05:50:34.371721+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:50:34.371721+0000","last_undegraded":"2024-05-08T05:50:34.371721+0000","last_fullsized":"2024-05-08T05:50:34.371721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307550+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:50:34.307550+0000","last_peered":"2024-05-08T05:50:34.307550+0000","last_clean":"2024-05-08T05:50:34.307550+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:50:34.307550+0000","last_undegraded":"2024-05-08T05:50:34.307550+0000","last_fullsized":"2024-05-08T05:50:34.307550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309201+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:50:34.309201+0000","last_peered":"2024-05-08T05:50:34.309201+0000","last_clean":"2024-05-08T05:50:34.309201+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:50:34.309201+0000","last_undegraded":"2024-05-08T05:50:34.309201+0000","last_fullsized":"2024-05-08T05:50:34.309201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306321+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:50:34.306321+0000","last_peered":"2024-05-08T05:50:34.306321+0000","last_clean":"2024-05-08T05:50:34.306321+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:50:34.306321+0000","last_undegraded":"2024-05-08T05:50:34.306321+0000","last_fullsized":"2024-05-08T05:50:34.306321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382473+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:50:34.382473+0000","last_peered":"2024-05-08T05:50:34.382473+0000","last_clean":"2024-05-08T05:50:34.382473+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:50:34.382473+0000","last_undegraded":"2024-05-08T05:50:34.382473+0000","last_fullsized":"2024-05-08T05:50:34.382473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309553+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:50:34.309553+0000","last_peered":"2024-05-08T05:50:34.309553+0000","last_clean":"2024-05-08T05:50:34.309553+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:50:34.309553+0000","last_undegraded":"2024-05-08T05:50:34.309553+0000","last_fullsized":"2024-05-08T05:50:34.309553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65925,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":94,"num_write":90,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":186969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":495642,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1141832,"data_compressed":62105,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":500320,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266737,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":496246,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462965,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1090688,"data_compressed":62105,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626491,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":491546,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":551270,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":491636,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":551304,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":491650,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":495664,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":1086192,"data_compressed":62105,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:30.524 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:51:30.525 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:30 smithi104 ceph-mon[34559]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.9 KiB/s wr, 5 op/s 2024-05-08T05:51:30.525 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:30 smithi104 ceph-mon[34559]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:51:30.782 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:32.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:32 smithi003 ceph-mon[30137]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.665+0000 7ff3f14b2700 1 -- 172.21.15.3:0/3129036211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1030b0 msgr2=0x7ff3ec10f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.665+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/3129036211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1030b0 0x7ff3ec10f230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400b0a0 tx=0x7ff3e400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 -- 172.21.15.3:0/3129036211 shutdown_connections 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/3129036211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1030b0 0x7ff3ec10f230 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/3129036211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3ec107420 0x7ff3ec107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/3129036211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 -- 172.21.15.3:0/3129036211 >> 172.21.15.3:0/3129036211 conn(0x7ff3ec078210 msgr2=0x7ff3ec078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 -- 172.21.15.3:0/3129036211 shutdown_connections 2024-05-08T05:51:32.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 -- 172.21.15.3:0/3129036211 wait complete. 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 Processor -- start 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.666+0000 7ff3f14b2700 1 -- start start 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3f14b2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3f14b2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3ec1030b0 0x7ff3ec198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3f14b2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ec107420 0x7ff3ec19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3f14b2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec112550 con 0x7ff3ec1026f0 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3f14b2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1123d0 con 0x7ff3ec107420 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3f14b2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3ec1126d0 con 0x7ff3ec1030b0 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36252/0 (socket says 172.21.15.3:36252) 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 -- 172.21.15.3:0/227128630 learned_addr learned my addr 172.21.15.3:0/227128630 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eb7fe700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ec107420 0x7ff3ec19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3e27fc700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3ec1030b0 0x7ff3ec198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 -- 172.21.15.3:0/227128630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3ec1030b0 msgr2=0x7ff3ec198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3ec1030b0 0x7ff3ec198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 -- 172.21.15.3:0/227128630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ec107420 msgr2=0x7ff3ec19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ec107420 0x7ff3ec19d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.667+0000 7ff3eaffd700 1 -- 172.21.15.3:0/227128630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3ec19dbb0 con 0x7ff3ec1026f0 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.668+0000 7ff3eaffd700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec198620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00d980 tx=0x7ff3dc00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.668+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc016070 con 0x7ff3ec1026f0 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.668+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3ec19dea0 con 0x7ff3ec1026f0 2024-05-08T05:51:32.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.668+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3dc00bb60 con 0x7ff3ec1026f0 2024-05-08T05:51:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.668+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3ec19e430 con 0x7ff3ec1026f0 2024-05-08T05:51:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.668+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc00f3c0 con 0x7ff3ec1026f0 2024-05-08T05:51:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.669+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc005360 con 0x7ff3ec1026f0 2024-05-08T05:51:32.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.671+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff3dc019070 con 0x7ff3ec1026f0 2024-05-08T05:51:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.672+0000 7ff3e8ff9700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff3d80b2990 0x7ff3d80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.672+0000 7ff3e27fc700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff3d80b2990 0x7ff3d80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:32.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.672+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7ff3dc0e2bb0 con 0x7ff3ec1026f0 2024-05-08T05:51:32.673 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.672+0000 7ff3e27fc700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff3d80b2990 0x7ff3d80b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4005950 tx=0x7ff3d40058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:32.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.675+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3dc020030 con 0x7ff3ec1026f0 2024-05-08T05:51:32.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:32 smithi089 ceph-mon[34495]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-05-08T05:51:32.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:32 smithi104 ceph-mon[34559]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-05-08T05:51:32.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.891+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3cc000bf0 con 0x7ff3d80b2990 2024-05-08T05:51:32.901 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.901+0000 7ff3e8ff9700 1 -- 172.21.15.3:0/227128630 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7ff3cc000bf0 con 0x7ff3d80b2990 2024-05-08T05:51:32.905 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff3d80b2990 msgr2=0x7ff3d80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff3d80b2990 0x7ff3d80b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4005950 tx=0x7ff3d40058e0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 msgr2=0x7ff3ec198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec198620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00d980 tx=0x7ff3dc00dd40 comp rx=0 tx=0).stop 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 shutdown_connections 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff3ec107420 0x7ff3ec19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff3d80b2990 0x7ff3d80b4e50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff3ec1026f0 0x7ff3ec198620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.908 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 --2- 172.21.15.3:0/227128630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3ec1030b0 0x7ff3ec198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:32.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 >> 172.21.15.3:0/227128630 conn(0x7ff3ec078210 msgr2=0x7ff3ec100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:32.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 shutdown_connections 2024-05-08T05:51:32.909 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:32.904+0000 7ff3f14b2700 1 -- 172.21.15.3:0/227128630 wait complete. 2024-05-08T05:51:32.909 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:51:33.712 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-05-08T05:51:31.237040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656527,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":147,"num_read_kb":163,"num_write":215,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360696,"kb_used_data":11528,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581128,"statfs":{"total":1151940427776,"available":1151571075072,"internally_reserved":0,"allocated":11804672,"data_stored":7883990,"data_compressed":424473,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9668,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":27,"num_write":12,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003340"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309094+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:50:34.309094+0000","last_peered":"2024-05-08T05:50:34.309094+0000","last_clean":"2024-05-08T05:50:34.309094+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:50:34.309094+0000","last_undegraded":"2024-05-08T05:50:34.309094+0000","last_fullsized":"2024-05-08T05:50:34.309094+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301929+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:50:34.301929+0000","last_peered":"2024-05-08T05:50:34.301929+0000","last_clean":"2024-05-08T05:50:34.301929+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:50:34.301929+0000","last_undegraded":"2024-05-08T05:50:34.301929+0000","last_fullsized":"2024-05-08T05:50:34.301929+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301712+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:50:34.301712+0000","last_peered":"2024-05-08T05:50:34.301712+0000","last_clean":"2024-05-08T05:50:34.301712+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:50:34.301712+0000","last_undegraded":"2024-05-08T05:50:34.301712+0000","last_fullsized":"2024-05-08T05:50:34.301712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299566+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:50:34.299566+0000","last_peered":"2024-05-08T05:50:34.299566+0000","last_clean":"2024-05-08T05:50:34.299566+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:50:34.299566+0000","last_undegraded":"2024-05-08T05:50:34.299566+0000","last_fullsized":"2024-05-08T05:50:34.299566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299857+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:50:34.299857+0000","last_peered":"2024-05-08T05:50:34.299857+0000","last_clean":"2024-05-08T05:50:34.299857+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:50:34.299857+0000","last_undegraded":"2024-05-08T05:50:34.299857+0000","last_fullsized":"2024-05-08T05:50:34.299857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302516+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:50:34.302516+0000","last_peered":"2024-05-08T05:50:34.302516+0000","last_clean":"2024-05-08T05:50:34.302516+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:50:34.302516+0000","last_undegraded":"2024-05-08T05:50:34.302516+0000","last_fullsized":"2024-05-08T05:50:34.302516+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493816+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:50:34.493816+0000","last_peered":"2024-05-08T05:50:34.493816+0000","last_clean":"2024-05-08T05:50:34.493816+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:50:34.493816+0000","last_undegraded":"2024-05-08T05:50:34.493816+0000","last_fullsized":"2024-05-08T05:50:34.493816+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371124+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:50:34.371124+0000","last_peered":"2024-05-08T05:50:34.371124+0000","last_clean":"2024-05-08T05:50:34.371124+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:50:34.371124+0000","last_undegraded":"2024-05-08T05:50:34.371124+0000","last_fullsized":"2024-05-08T05:50:34.371124+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306446+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:50:34.306446+0000","last_peered":"2024-05-08T05:50:34.306446+0000","last_clean":"2024-05-08T05:50:34.306446+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:50:34.306446+0000","last_undegraded":"2024-05-08T05:50:34.306446+0000","last_fullsized":"2024-05-08T05:50:34.306446+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302327+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:50:34.302327+0000","last_peered":"2024-05-08T05:50:34.302327+0000","last_clean":"2024-05-08T05:50:34.302327+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:50:34.302327+0000","last_undegraded":"2024-05-08T05:50:34.302327+0000","last_fullsized":"2024-05-08T05:50:34.302327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494192+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:50:34.494192+0000","last_peered":"2024-05-08T05:50:34.494192+0000","last_clean":"2024-05-08T05:50:34.494192+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:50:34.494192+0000","last_undegraded":"2024-05-08T05:50:34.494192+0000","last_fullsized":"2024-05-08T05:50:34.494192+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299489+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:50:34.299489+0000","last_peered":"2024-05-08T05:50:34.299489+0000","last_clean":"2024-05-08T05:50:34.299489+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:50:34.299489+0000","last_undegraded":"2024-05-08T05:50:34.299489+0000","last_fullsized":"2024-05-08T05:50:34.299489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382579+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:50:34.382579+0000","last_peered":"2024-05-08T05:50:34.382579+0000","last_clean":"2024-05-08T05:50:34.382579+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:50:34.382579+0000","last_undegraded":"2024-05-08T05:50:34.382579+0000","last_fullsized":"2024-05-08T05:50:34.382579+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301354+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:50:34.301354+0000","last_peered":"2024-05-08T05:50:34.301354+0000","last_clean":"2024-05-08T05:50:34.301354+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:50:34.301354+0000","last_undegraded":"2024-05-08T05:50:34.301354+0000","last_fullsized":"2024-05-08T05:50:34.301354+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301406+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:50:34.301406+0000","last_peered":"2024-05-08T05:50:34.301406+0000","last_clean":"2024-05-08T05:50:34.301406+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:50:34.301406+0000","last_undegraded":"2024-05-08T05:50:34.301406+0000","last_fullsized":"2024-05-08T05:50:34.301406+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305522+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:50:34.305522+0000","last_peered":"2024-05-08T05:50:34.305522+0000","last_clean":"2024-05-08T05:50:34.305522+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:50:34.305522+0000","last_undegraded":"2024-05-08T05:50:34.305522+0000","last_fullsized":"2024-05-08T05:50:34.305522+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308757+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:50:34.308757+0000","last_peered":"2024-05-08T05:50:34.308757+0000","last_clean":"2024-05-08T05:50:34.308757+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:50:34.308757+0000","last_undegraded":"2024-05-08T05:50:34.308757+0000","last_fullsized":"2024-05-08T05:50:34.308757+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305027+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:50:34.305027+0000","last_peered":"2024-05-08T05:50:34.305027+0000","last_clean":"2024-05-08T05:50:34.305027+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:50:34.305027+0000","last_undegraded":"2024-05-08T05:50:34.305027+0000","last_fullsized":"2024-05-08T05:50:34.305027+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304350+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:50:34.304350+0000","last_peered":"2024-05-08T05:50:34.304350+0000","last_clean":"2024-05-08T05:50:34.304350+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:50:34.304350+0000","last_undegraded":"2024-05-08T05:50:34.304350+0000","last_fullsized":"2024-05-08T05:50:34.304350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302774+0000","last_change":"2024-05-08T05:50:12.705411+0000","last_active":"2024-05-08T05:50:34.302774+0000","last_peered":"2024-05-08T05:50:34.302774+0000","last_clean":"2024-05-08T05:50:34.302774+0000","last_became_active":"2024-05-08T05:50:12.704966+0000","last_became_peered":"2024-05-08T05:50:12.704966+0000","last_unstale":"2024-05-08T05:50:34.302774+0000","last_undegraded":"2024-05-08T05:50:34.302774+0000","last_fullsized":"2024-05-08T05:50:34.302774+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:35.286238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305173+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:50:34.305173+0000","last_peered":"2024-05-08T05:50:34.305173+0000","last_clean":"2024-05-08T05:50:34.305173+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:50:34.305173+0000","last_undegraded":"2024-05-08T05:50:34.305173+0000","last_fullsized":"2024-05-08T05:50:34.305173+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371532+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:50:34.371532+0000","last_peered":"2024-05-08T05:50:34.371532+0000","last_clean":"2024-05-08T05:50:34.371532+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:50:34.371532+0000","last_undegraded":"2024-05-08T05:50:34.371532+0000","last_fullsized":"2024-05-08T05:50:34.371532+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299072+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:50:34.299072+0000","last_peered":"2024-05-08T05:50:34.299072+0000","last_clean":"2024-05-08T05:50:34.299072+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:50:34.299072+0000","last_undegraded":"2024-05-08T05:50:34.299072+0000","last_fullsized":"2024-05-08T05:50:34.299072+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371651+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:50:34.371651+0000","last_peered":"2024-05-08T05:50:34.371651+0000","last_clean":"2024-05-08T05:50:34.371651+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:50:34.371651+0000","last_undegraded":"2024-05-08T05:50:34.371651+0000","last_fullsized":"2024-05-08T05:50:34.371651+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302007+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:50:34.302007+0000","last_peered":"2024-05-08T05:50:34.302007+0000","last_clean":"2024-05-08T05:50:34.302007+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:50:34.302007+0000","last_undegraded":"2024-05-08T05:50:34.302007+0000","last_fullsized":"2024-05-08T05:50:34.302007+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299245+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:50:34.299245+0000","last_peered":"2024-05-08T05:50:34.299245+0000","last_clean":"2024-05-08T05:50:34.299245+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:50:34.299245+0000","last_undegraded":"2024-05-08T05:50:34.299245+0000","last_fullsized":"2024-05-08T05:50:34.299245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300482+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:50:34.300482+0000","last_peered":"2024-05-08T05:50:34.300482+0000","last_clean":"2024-05-08T05:50:34.300482+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:50:34.300482+0000","last_undegraded":"2024-05-08T05:50:34.300482+0000","last_fullsized":"2024-05-08T05:50:34.300482+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306136+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:50:34.306136+0000","last_peered":"2024-05-08T05:50:34.306136+0000","last_clean":"2024-05-08T05:50:34.306136+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:50:34.306136+0000","last_undegraded":"2024-05-08T05:50:34.306136+0000","last_fullsized":"2024-05-08T05:50:34.306136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299191+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:50:34.299191+0000","last_peered":"2024-05-08T05:50:34.299191+0000","last_clean":"2024-05-08T05:50:34.299191+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:50:34.299191+0000","last_undegraded":"2024-05-08T05:50:34.299191+0000","last_fullsized":"2024-05-08T05:50:34.299191+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.314163+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:50:34.314163+0000","last_peered":"2024-05-08T05:50:34.314163+0000","last_clean":"2024-05-08T05:50:34.314163+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:50:34.314163+0000","last_undegraded":"2024-05-08T05:50:34.314163+0000","last_fullsized":"2024-05-08T05:50:34.314163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.383855+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:50:34.383855+0000","last_peered":"2024-05-08T05:50:34.383855+0000","last_clean":"2024-05-08T05:50:34.383855+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:50:34.383855+0000","last_undegraded":"2024-05-08T05:50:34.383855+0000","last_fullsized":"2024-05-08T05:50:34.383855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304090+0000","last_change":"2024-05-08T05:50:12.707786+0000","last_active":"2024-05-08T05:50:34.304090+0000","last_peered":"2024-05-08T05:50:34.304090+0000","last_clean":"2024-05-08T05:50:34.304090+0000","last_became_active":"2024-05-08T05:50:12.707349+0000","last_became_peered":"2024-05-08T05:50:12.707349+0000","last_unstale":"2024-05-08T05:50:34.304090+0000","last_undegraded":"2024-05-08T05:50:34.304090+0000","last_fullsized":"2024-05-08T05:50:34.304090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:52.337370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304616+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:50:34.304616+0000","last_peered":"2024-05-08T05:50:34.304616+0000","last_clean":"2024-05-08T05:50:34.304616+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:50:34.304616+0000","last_undegraded":"2024-05-08T05:50:34.304616+0000","last_fullsized":"2024-05-08T05:50:34.304616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299828+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:50:34.299828+0000","last_peered":"2024-05-08T05:50:34.299828+0000","last_clean":"2024-05-08T05:50:34.299828+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:50:34.299828+0000","last_undegraded":"2024-05-08T05:50:34.299828+0000","last_fullsized":"2024-05-08T05:50:34.299828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303784+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:50:34.303784+0000","last_peered":"2024-05-08T05:50:34.303784+0000","last_clean":"2024-05-08T05:50:34.303784+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:50:34.303784+0000","last_undegraded":"2024-05-08T05:50:34.303784+0000","last_fullsized":"2024-05-08T05:50:34.303784+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":41,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302445+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:50:34.302445+0000","last_peered":"2024-05-08T05:50:34.302445+0000","last_clean":"2024-05-08T05:50:34.302445+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:50:34.302445+0000","last_undegraded":"2024-05-08T05:50:34.302445+0000","last_fullsized":"2024-05-08T05:50:34.302445+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.373044+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:50:34.373044+0000","last_peered":"2024-05-08T05:50:34.373044+0000","last_clean":"2024-05-08T05:50:34.373044+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:50:34.373044+0000","last_undegraded":"2024-05-08T05:50:34.373044+0000","last_fullsized":"2024-05-08T05:50:34.373044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308643+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:50:34.308643+0000","last_peered":"2024-05-08T05:50:34.308643+0000","last_clean":"2024-05-08T05:50:34.308643+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:50:34.308643+0000","last_undegraded":"2024-05-08T05:50:34.308643+0000","last_fullsized":"2024-05-08T05:50:34.308643+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382236+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:50:34.382236+0000","last_peered":"2024-05-08T05:50:34.382236+0000","last_clean":"2024-05-08T05:50:34.382236+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:50:34.382236+0000","last_undegraded":"2024-05-08T05:50:34.382236+0000","last_fullsized":"2024-05-08T05:50:34.382236+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371244+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:50:34.371244+0000","last_peered":"2024-05-08T05:50:34.371244+0000","last_clean":"2024-05-08T05:50:34.371244+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:50:34.371244+0000","last_undegraded":"2024-05-08T05:50:34.371244+0000","last_fullsized":"2024-05-08T05:50:34.371244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301740+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:50:34.301740+0000","last_peered":"2024-05-08T05:50:34.301740+0000","last_clean":"2024-05-08T05:50:34.301740+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:50:34.301740+0000","last_undegraded":"2024-05-08T05:50:34.301740+0000","last_fullsized":"2024-05-08T05:50:34.301740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304326+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:50:34.304326+0000","last_peered":"2024-05-08T05:50:34.304326+0000","last_clean":"2024-05-08T05:50:34.304326+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:50:34.304326+0000","last_undegraded":"2024-05-08T05:50:34.304326+0000","last_fullsized":"2024-05-08T05:50:34.304326+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304401+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:50:34.304401+0000","last_peered":"2024-05-08T05:50:34.304401+0000","last_clean":"2024-05-08T05:50:34.304401+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:50:34.304401+0000","last_undegraded":"2024-05-08T05:50:34.304401+0000","last_fullsized":"2024-05-08T05:50:34.304401+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371810+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:50:34.371810+0000","last_peered":"2024-05-08T05:50:34.371810+0000","last_clean":"2024-05-08T05:50:34.371810+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:50:34.371810+0000","last_undegraded":"2024-05-08T05:50:34.371810+0000","last_fullsized":"2024-05-08T05:50:34.371810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302379+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:50:34.302379+0000","last_peered":"2024-05-08T05:50:34.302379+0000","last_clean":"2024-05-08T05:50:34.302379+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:50:34.302379+0000","last_undegraded":"2024-05-08T05:50:34.302379+0000","last_fullsized":"2024-05-08T05:50:34.302379+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306647+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:50:34.306647+0000","last_peered":"2024-05-08T05:50:34.306647+0000","last_clean":"2024-05-08T05:50:34.306647+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:50:34.306647+0000","last_undegraded":"2024-05-08T05:50:34.306647+0000","last_fullsized":"2024-05-08T05:50:34.306647+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303999+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:50:34.303999+0000","last_peered":"2024-05-08T05:50:34.303999+0000","last_clean":"2024-05-08T05:50:34.303999+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:50:34.303999+0000","last_undegraded":"2024-05-08T05:50:34.303999+0000","last_fullsized":"2024-05-08T05:50:34.303999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494225+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:50:34.494225+0000","last_peered":"2024-05-08T05:50:34.494225+0000","last_clean":"2024-05-08T05:50:34.494225+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:50:34.494225+0000","last_undegraded":"2024-05-08T05:50:34.494225+0000","last_fullsized":"2024-05-08T05:50:34.494225+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301971+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:50:34.301971+0000","last_peered":"2024-05-08T05:50:34.301971+0000","last_clean":"2024-05-08T05:50:34.301971+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:50:34.301971+0000","last_undegraded":"2024-05-08T05:50:34.301971+0000","last_fullsized":"2024-05-08T05:50:34.301971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299540+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:50:34.299540+0000","last_peered":"2024-05-08T05:50:34.299540+0000","last_clean":"2024-05-08T05:50:34.299540+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:50:34.299540+0000","last_undegraded":"2024-05-08T05:50:34.299540+0000","last_fullsized":"2024-05-08T05:50:34.299540+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304053+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:50:34.304053+0000","last_peered":"2024-05-08T05:50:34.304053+0000","last_clean":"2024-05-08T05:50:34.304053+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:50:34.304053+0000","last_undegraded":"2024-05-08T05:50:34.304053+0000","last_fullsized":"2024-05-08T05:50:34.304053+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305890+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:50:34.305890+0000","last_peered":"2024-05-08T05:50:34.305890+0000","last_clean":"2024-05-08T05:50:34.305890+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:50:34.305890+0000","last_undegraded":"2024-05-08T05:50:34.305890+0000","last_fullsized":"2024-05-08T05:50:34.305890+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308283+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:50:34.308283+0000","last_peered":"2024-05-08T05:50:34.308283+0000","last_clean":"2024-05-08T05:50:34.308283+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:50:34.308283+0000","last_undegraded":"2024-05-08T05:50:34.308283+0000","last_fullsized":"2024-05-08T05:50:34.308283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:51:28.493623+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:51:28.493623+0000","last_peered":"2024-05-08T05:51:28.493623+0000","last_clean":"2024-05-08T05:51:28.493623+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:51:28.493623+0000","last_undegraded":"2024-05-08T05:51:28.493623+0000","last_fullsized":"2024-05-08T05:51:28.493623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65127,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":60,"num_read_kb":88,"num_write":33,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300373+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:50:34.300373+0000","last_peered":"2024-05-08T05:50:34.300373+0000","last_clean":"2024-05-08T05:50:34.300373+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:50:34.300373+0000","last_undegraded":"2024-05-08T05:50:34.300373+0000","last_fullsized":"2024-05-08T05:50:34.300373+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305497+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:50:34.305497+0000","last_peered":"2024-05-08T05:50:34.305497+0000","last_clean":"2024-05-08T05:50:34.305497+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:50:34.305497+0000","last_undegraded":"2024-05-08T05:50:34.305497+0000","last_fullsized":"2024-05-08T05:50:34.305497+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303591+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:50:34.303591+0000","last_peered":"2024-05-08T05:50:34.303591+0000","last_clean":"2024-05-08T05:50:34.303591+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:50:34.303591+0000","last_undegraded":"2024-05-08T05:50:34.303591+0000","last_fullsized":"2024-05-08T05:50:34.303591+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382278+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:50:34.382278+0000","last_peered":"2024-05-08T05:50:34.382278+0000","last_clean":"2024-05-08T05:50:34.382278+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:50:34.382278+0000","last_undegraded":"2024-05-08T05:50:34.382278+0000","last_fullsized":"2024-05-08T05:50:34.382278+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309374+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:50:34.309374+0000","last_peered":"2024-05-08T05:50:34.309374+0000","last_clean":"2024-05-08T05:50:34.309374+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:50:34.309374+0000","last_undegraded":"2024-05-08T05:50:34.309374+0000","last_fullsized":"2024-05-08T05:50:34.309374+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305212+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:50:34.305212+0000","last_peered":"2024-05-08T05:50:34.305212+0000","last_clean":"2024-05-08T05:50:34.305212+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:50:34.305212+0000","last_undegraded":"2024-05-08T05:50:34.305212+0000","last_fullsized":"2024-05-08T05:50:34.305212+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493970+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:50:34.493970+0000","last_peered":"2024-05-08T05:50:34.493970+0000","last_clean":"2024-05-08T05:50:34.493970+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:50:34.493970+0000","last_undegraded":"2024-05-08T05:50:34.493970+0000","last_fullsized":"2024-05-08T05:50:34.493970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304887+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:50:34.304887+0000","last_peered":"2024-05-08T05:50:34.304887+0000","last_clean":"2024-05-08T05:50:34.304887+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:50:34.304887+0000","last_undegraded":"2024-05-08T05:50:34.304887+0000","last_fullsized":"2024-05-08T05:50:34.304887+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299136+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:50:34.299136+0000","last_peered":"2024-05-08T05:50:34.299136+0000","last_clean":"2024-05-08T05:50:34.299136+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:50:34.299136+0000","last_undegraded":"2024-05-08T05:50:34.299136+0000","last_fullsized":"2024-05-08T05:50:34.299136+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493834+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:50:34.493834+0000","last_peered":"2024-05-08T05:50:34.493834+0000","last_clean":"2024-05-08T05:50:34.493834+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:50:34.493834+0000","last_undegraded":"2024-05-08T05:50:34.493834+0000","last_fullsized":"2024-05-08T05:50:34.493834+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382601+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:50:34.382601+0000","last_peered":"2024-05-08T05:50:34.382601+0000","last_clean":"2024-05-08T05:50:34.382601+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:50:34.382601+0000","last_undegraded":"2024-05-08T05:50:34.382601+0000","last_fullsized":"2024-05-08T05:50:34.382601+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.315867+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:50:34.315867+0000","last_peered":"2024-05-08T05:50:34.315867+0000","last_clean":"2024-05-08T05:50:34.315867+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:50:34.315867+0000","last_undegraded":"2024-05-08T05:50:34.315867+0000","last_fullsized":"2024-05-08T05:50:34.315867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306536+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:50:34.306536+0000","last_peered":"2024-05-08T05:50:34.306536+0000","last_clean":"2024-05-08T05:50:34.306536+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:50:34.306536+0000","last_undegraded":"2024-05-08T05:50:34.306536+0000","last_fullsized":"2024-05-08T05:50:34.306536+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306413+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:50:34.306413+0000","last_peered":"2024-05-08T05:50:34.306413+0000","last_clean":"2024-05-08T05:50:34.306413+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:50:34.306413+0000","last_undegraded":"2024-05-08T05:50:34.306413+0000","last_fullsized":"2024-05-08T05:50:34.306413+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300797+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:50:34.300797+0000","last_peered":"2024-05-08T05:50:34.300797+0000","last_clean":"2024-05-08T05:50:34.300797+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:50:34.300797+0000","last_undegraded":"2024-05-08T05:50:34.300797+0000","last_fullsized":"2024-05-08T05:50:34.300797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.310426+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:50:34.310426+0000","last_peered":"2024-05-08T05:50:34.310426+0000","last_clean":"2024-05-08T05:50:34.310426+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:50:34.310426+0000","last_undegraded":"2024-05-08T05:50:34.310426+0000","last_fullsized":"2024-05-08T05:50:34.310426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309753+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:50:34.309753+0000","last_peered":"2024-05-08T05:50:34.309753+0000","last_clean":"2024-05-08T05:50:34.309753+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:50:34.309753+0000","last_undegraded":"2024-05-08T05:50:34.309753+0000","last_fullsized":"2024-05-08T05:50:34.309753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.312120+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:50:34.312120+0000","last_peered":"2024-05-08T05:50:34.312120+0000","last_clean":"2024-05-08T05:50:34.312120+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:50:34.312120+0000","last_undegraded":"2024-05-08T05:50:34.312120+0000","last_fullsized":"2024-05-08T05:50:34.312120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301983+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:50:34.301983+0000","last_peered":"2024-05-08T05:50:34.301983+0000","last_clean":"2024-05-08T05:50:34.301983+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:50:34.301983+0000","last_undegraded":"2024-05-08T05:50:34.301983+0000","last_fullsized":"2024-05-08T05:50:34.301983+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382552+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:50:34.382552+0000","last_peered":"2024-05-08T05:50:34.382552+0000","last_clean":"2024-05-08T05:50:34.382552+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:50:34.382552+0000","last_undegraded":"2024-05-08T05:50:34.382552+0000","last_fullsized":"2024-05-08T05:50:34.382552+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493846+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:50:34.493846+0000","last_peered":"2024-05-08T05:50:34.493846+0000","last_clean":"2024-05-08T05:50:34.493846+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:50:34.493846+0000","last_undegraded":"2024-05-08T05:50:34.493846+0000","last_fullsized":"2024-05-08T05:50:34.493846+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306858+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:50:34.306858+0000","last_peered":"2024-05-08T05:50:34.306858+0000","last_clean":"2024-05-08T05:50:34.306858+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:50:34.306858+0000","last_undegraded":"2024-05-08T05:50:34.306858+0000","last_fullsized":"2024-05-08T05:50:34.306858+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382650+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:50:34.382650+0000","last_peered":"2024-05-08T05:50:34.382650+0000","last_clean":"2024-05-08T05:50:34.382650+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:50:34.382650+0000","last_undegraded":"2024-05-08T05:50:34.382650+0000","last_fullsized":"2024-05-08T05:50:34.382650+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305055+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:50:34.305055+0000","last_peered":"2024-05-08T05:50:34.305055+0000","last_clean":"2024-05-08T05:50:34.305055+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:50:34.305055+0000","last_undegraded":"2024-05-08T05:50:34.305055+0000","last_fullsized":"2024-05-08T05:50:34.305055+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382134+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:50:34.382134+0000","last_peered":"2024-05-08T05:50:34.382134+0000","last_clean":"2024-05-08T05:50:34.382134+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:50:34.382134+0000","last_undegraded":"2024-05-08T05:50:34.382134+0000","last_fullsized":"2024-05-08T05:50:34.382134+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494165+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:50:34.494165+0000","last_peered":"2024-05-08T05:50:34.494165+0000","last_clean":"2024-05-08T05:50:34.494165+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:50:34.494165+0000","last_undegraded":"2024-05-08T05:50:34.494165+0000","last_fullsized":"2024-05-08T05:50:34.494165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371490+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:50:34.371490+0000","last_peered":"2024-05-08T05:50:34.371490+0000","last_clean":"2024-05-08T05:50:34.371490+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:50:34.371490+0000","last_undegraded":"2024-05-08T05:50:34.371490+0000","last_fullsized":"2024-05-08T05:50:34.371490+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299546+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:50:34.299546+0000","last_peered":"2024-05-08T05:50:34.299546+0000","last_clean":"2024-05-08T05:50:34.299546+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:50:34.299546+0000","last_undegraded":"2024-05-08T05:50:34.299546+0000","last_fullsized":"2024-05-08T05:50:34.299546+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":115,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493747+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:50:34.493747+0000","last_peered":"2024-05-08T05:50:34.493747+0000","last_clean":"2024-05-08T05:50:34.493747+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:50:34.493747+0000","last_undegraded":"2024-05-08T05:50:34.493747+0000","last_fullsized":"2024-05-08T05:50:34.493747+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304375+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:50:34.304375+0000","last_peered":"2024-05-08T05:50:34.304375+0000","last_clean":"2024-05-08T05:50:34.304375+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:50:34.304375+0000","last_undegraded":"2024-05-08T05:50:34.304375+0000","last_fullsized":"2024-05-08T05:50:34.304375+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382105+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:50:34.382105+0000","last_peered":"2024-05-08T05:50:34.382105+0000","last_clean":"2024-05-08T05:50:34.382105+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:50:34.382105+0000","last_undegraded":"2024-05-08T05:50:34.382105+0000","last_fullsized":"2024-05-08T05:50:34.382105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493827+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:50:34.493827+0000","last_peered":"2024-05-08T05:50:34.493827+0000","last_clean":"2024-05-08T05:50:34.493827+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:50:34.493827+0000","last_undegraded":"2024-05-08T05:50:34.493827+0000","last_fullsized":"2024-05-08T05:50:34.493827+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307301+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:50:34.307301+0000","last_peered":"2024-05-08T05:50:34.307301+0000","last_clean":"2024-05-08T05:50:34.307301+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:50:34.307301+0000","last_undegraded":"2024-05-08T05:50:34.307301+0000","last_fullsized":"2024-05-08T05:50:34.307301+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300158+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:50:34.300158+0000","last_peered":"2024-05-08T05:50:34.300158+0000","last_clean":"2024-05-08T05:50:34.300158+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:50:34.300158+0000","last_undegraded":"2024-05-08T05:50:34.300158+0000","last_fullsized":"2024-05-08T05:50:34.300158+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382811+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:50:34.382811+0000","last_peered":"2024-05-08T05:50:34.382811+0000","last_clean":"2024-05-08T05:50:34.382811+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:50:34.382811+0000","last_undegraded":"2024-05-08T05:50:34.382811+0000","last_fullsized":"2024-05-08T05:50:34.382811+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371130+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:50:34.371130+0000","last_peered":"2024-05-08T05:50:34.371130+0000","last_clean":"2024-05-08T05:50:34.371130+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:50:34.371130+0000","last_undegraded":"2024-05-08T05:50:34.371130+0000","last_fullsized":"2024-05-08T05:50:34.371130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":50,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:51:22.655968+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:51:22.655968+0000","last_peered":"2024-05-08T05:51:22.655968+0000","last_clean":"2024-05-08T05:51:22.655968+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:51:22.655968+0000","last_undegraded":"2024-05-08T05:51:22.655968+0000","last_fullsized":"2024-05-08T05:51:22.655968+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371519+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:50:34.371519+0000","last_peered":"2024-05-08T05:50:34.371519+0000","last_clean":"2024-05-08T05:50:34.371519+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:50:34.371519+0000","last_undegraded":"2024-05-08T05:50:34.371519+0000","last_fullsized":"2024-05-08T05:50:34.371519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302619+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:50:34.302619+0000","last_peered":"2024-05-08T05:50:34.302619+0000","last_clean":"2024-05-08T05:50:34.302619+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:50:34.302619+0000","last_undegraded":"2024-05-08T05:50:34.302619+0000","last_fullsized":"2024-05-08T05:50:34.302619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.300507+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:50:34.300507+0000","last_peered":"2024-05-08T05:50:34.300507+0000","last_clean":"2024-05-08T05:50:34.300507+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:50:34.300507+0000","last_undegraded":"2024-05-08T05:50:34.300507+0000","last_fullsized":"2024-05-08T05:50:34.300507+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.302069+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:50:34.302069+0000","last_peered":"2024-05-08T05:50:34.302069+0000","last_clean":"2024-05-08T05:50:34.302069+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:50:34.302069+0000","last_undegraded":"2024-05-08T05:50:34.302069+0000","last_fullsized":"2024-05-08T05:50:34.302069+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299472+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:50:34.299472+0000","last_peered":"2024-05-08T05:50:34.299472+0000","last_clean":"2024-05-08T05:50:34.299472+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:50:34.299472+0000","last_undegraded":"2024-05-08T05:50:34.299472+0000","last_fullsized":"2024-05-08T05:50:34.299472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305592+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:50:34.305592+0000","last_peered":"2024-05-08T05:50:34.305592+0000","last_clean":"2024-05-08T05:50:34.305592+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:50:34.305592+0000","last_undegraded":"2024-05-08T05:50:34.305592+0000","last_fullsized":"2024-05-08T05:50:34.305592+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299749+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:50:34.299749+0000","last_peered":"2024-05-08T05:50:34.299749+0000","last_clean":"2024-05-08T05:50:34.299749+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:50:34.299749+0000","last_undegraded":"2024-05-08T05:50:34.299749+0000","last_fullsized":"2024-05-08T05:50:34.299749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304873+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:50:34.304873+0000","last_peered":"2024-05-08T05:50:34.304873+0000","last_clean":"2024-05-08T05:50:34.304873+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:50:34.304873+0000","last_undegraded":"2024-05-08T05:50:34.304873+0000","last_fullsized":"2024-05-08T05:50:34.304873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371403+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:50:34.371403+0000","last_peered":"2024-05-08T05:50:34.371403+0000","last_clean":"2024-05-08T05:50:34.371403+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:50:34.371403+0000","last_undegraded":"2024-05-08T05:50:34.371403+0000","last_fullsized":"2024-05-08T05:50:34.371403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307337+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:50:34.307337+0000","last_peered":"2024-05-08T05:50:34.307337+0000","last_clean":"2024-05-08T05:50:34.307337+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:50:34.307337+0000","last_undegraded":"2024-05-08T05:50:34.307337+0000","last_fullsized":"2024-05-08T05:50:34.307337+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299422+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:50:34.299422+0000","last_peered":"2024-05-08T05:50:34.299422+0000","last_clean":"2024-05-08T05:50:34.299422+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:50:34.299422+0000","last_undegraded":"2024-05-08T05:50:34.299422+0000","last_fullsized":"2024-05-08T05:50:34.299422+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303161+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:50:34.303161+0000","last_peered":"2024-05-08T05:50:34.303161+0000","last_clean":"2024-05-08T05:50:34.303161+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:50:34.303161+0000","last_undegraded":"2024-05-08T05:50:34.303161+0000","last_fullsized":"2024-05-08T05:50:34.303161+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371523+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:50:34.371523+0000","last_peered":"2024-05-08T05:50:34.371523+0000","last_clean":"2024-05-08T05:50:34.371523+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:50:34.371523+0000","last_undegraded":"2024-05-08T05:50:34.371523+0000","last_fullsized":"2024-05-08T05:50:34.371523+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299549+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:50:34.299549+0000","last_peered":"2024-05-08T05:50:34.299549+0000","last_clean":"2024-05-08T05:50:34.299549+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:50:34.299549+0000","last_undegraded":"2024-05-08T05:50:34.299549+0000","last_fullsized":"2024-05-08T05:50:34.299549+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.299998+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:50:34.299998+0000","last_peered":"2024-05-08T05:50:34.299998+0000","last_clean":"2024-05-08T05:50:34.299998+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:50:34.299998+0000","last_undegraded":"2024-05-08T05:50:34.299998+0000","last_fullsized":"2024-05-08T05:50:34.299998+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494039+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:50:34.494039+0000","last_peered":"2024-05-08T05:50:34.494039+0000","last_clean":"2024-05-08T05:50:34.494039+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:50:34.494039+0000","last_undegraded":"2024-05-08T05:50:34.494039+0000","last_fullsized":"2024-05-08T05:50:34.494039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.305715+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:50:34.305715+0000","last_peered":"2024-05-08T05:50:34.305715+0000","last_clean":"2024-05-08T05:50:34.305715+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:50:34.305715+0000","last_undegraded":"2024-05-08T05:50:34.305715+0000","last_fullsized":"2024-05-08T05:50:34.305715+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304866+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:50:34.304866+0000","last_peered":"2024-05-08T05:50:34.304866+0000","last_clean":"2024-05-08T05:50:34.304866+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:50:34.304866+0000","last_undegraded":"2024-05-08T05:50:34.304866+0000","last_fullsized":"2024-05-08T05:50:34.304866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494288+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:50:34.494288+0000","last_peered":"2024-05-08T05:50:34.494288+0000","last_clean":"2024-05-08T05:50:34.494288+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:50:34.494288+0000","last_undegraded":"2024-05-08T05:50:34.494288+0000","last_fullsized":"2024-05-08T05:50:34.494288+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301345+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:50:34.301345+0000","last_peered":"2024-05-08T05:50:34.301345+0000","last_clean":"2024-05-08T05:50:34.301345+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:50:34.301345+0000","last_undegraded":"2024-05-08T05:50:34.301345+0000","last_fullsized":"2024-05-08T05:50:34.301345+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304249+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:50:34.304249+0000","last_peered":"2024-05-08T05:50:34.304249+0000","last_clean":"2024-05-08T05:50:34.304249+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:50:34.304249+0000","last_undegraded":"2024-05-08T05:50:34.304249+0000","last_fullsized":"2024-05-08T05:50:34.304249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.493978+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:50:34.493978+0000","last_peered":"2024-05-08T05:50:34.493978+0000","last_clean":"2024-05-08T05:50:34.493978+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:50:34.493978+0000","last_undegraded":"2024-05-08T05:50:34.493978+0000","last_fullsized":"2024-05-08T05:50:34.493978+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304117+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:50:34.304117+0000","last_peered":"2024-05-08T05:50:34.304117+0000","last_clean":"2024-05-08T05:50:34.304117+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:50:34.304117+0000","last_undegraded":"2024-05-08T05:50:34.304117+0000","last_fullsized":"2024-05-08T05:50:34.304117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":42,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382642+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:50:34.382642+0000","last_peered":"2024-05-08T05:50:34.382642+0000","last_clean":"2024-05-08T05:50:34.382642+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:50:34.382642+0000","last_undegraded":"2024-05-08T05:50:34.382642+0000","last_fullsized":"2024-05-08T05:50:34.382642+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.494351+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:50:34.494351+0000","last_peered":"2024-05-08T05:50:34.494351+0000","last_clean":"2024-05-08T05:50:34.494351+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:50:34.494351+0000","last_undegraded":"2024-05-08T05:50:34.494351+0000","last_fullsized":"2024-05-08T05:50:34.494351+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307427+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:50:34.307427+0000","last_peered":"2024-05-08T05:50:34.307427+0000","last_clean":"2024-05-08T05:50:34.307427+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:50:34.307427+0000","last_undegraded":"2024-05-08T05:50:34.307427+0000","last_fullsized":"2024-05-08T05:50:34.307427+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303042+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:50:34.303042+0000","last_peered":"2024-05-08T05:50:34.303042+0000","last_clean":"2024-05-08T05:50:34.303042+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:50:34.303042+0000","last_undegraded":"2024-05-08T05:50:34.303042+0000","last_fullsized":"2024-05-08T05:50:34.303042+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.303033+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:50:34.303033+0000","last_peered":"2024-05-08T05:50:34.303033+0000","last_clean":"2024-05-08T05:50:34.303033+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:50:34.303033+0000","last_undegraded":"2024-05-08T05:50:34.303033+0000","last_fullsized":"2024-05-08T05:50:34.303033+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.301837+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:50:34.301837+0000","last_peered":"2024-05-08T05:50:34.301837+0000","last_clean":"2024-05-08T05:50:34.301837+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:50:34.301837+0000","last_undegraded":"2024-05-08T05:50:34.301837+0000","last_fullsized":"2024-05-08T05:50:34.301837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382697+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:50:34.382697+0000","last_peered":"2024-05-08T05:50:34.382697+0000","last_clean":"2024-05-08T05:50:34.382697+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:50:34.382697+0000","last_undegraded":"2024-05-08T05:50:34.382697+0000","last_fullsized":"2024-05-08T05:50:34.382697+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.308345+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:50:34.308345+0000","last_peered":"2024-05-08T05:50:34.308345+0000","last_clean":"2024-05-08T05:50:34.308345+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:50:34.308345+0000","last_undegraded":"2024-05-08T05:50:34.308345+0000","last_fullsized":"2024-05-08T05:50:34.308345+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.304732+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:50:34.304732+0000","last_peered":"2024-05-08T05:50:34.304732+0000","last_clean":"2024-05-08T05:50:34.304732+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:50:34.304732+0000","last_undegraded":"2024-05-08T05:50:34.304732+0000","last_fullsized":"2024-05-08T05:50:34.304732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.371721+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:50:34.371721+0000","last_peered":"2024-05-08T05:50:34.371721+0000","last_clean":"2024-05-08T05:50:34.371721+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:50:34.371721+0000","last_undegraded":"2024-05-08T05:50:34.371721+0000","last_fullsized":"2024-05-08T05:50:34.371721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.307550+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:50:34.307550+0000","last_peered":"2024-05-08T05:50:34.307550+0000","last_clean":"2024-05-08T05:50:34.307550+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:50:34.307550+0000","last_undegraded":"2024-05-08T05:50:34.307550+0000","last_fullsized":"2024-05-08T05:50:34.307550+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309201+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:50:34.309201+0000","last_peered":"2024-05-08T05:50:34.309201+0000","last_clean":"2024-05-08T05:50:34.309201+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:50:34.309201+0000","last_undegraded":"2024-05-08T05:50:34.309201+0000","last_fullsized":"2024-05-08T05:50:34.309201+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.306321+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:50:34.306321+0000","last_peered":"2024-05-08T05:50:34.306321+0000","last_clean":"2024-05-08T05:50:34.306321+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:50:34.306321+0000","last_undegraded":"2024-05-08T05:50:34.306321+0000","last_fullsized":"2024-05-08T05:50:34.306321+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.382473+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:50:34.382473+0000","last_peered":"2024-05-08T05:50:34.382473+0000","last_clean":"2024-05-08T05:50:34.382473+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:50:34.382473+0000","last_undegraded":"2024-05-08T05:50:34.382473+0000","last_fullsized":"2024-05-08T05:50:34.382473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-05-08T05:50:34.309553+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:50:34.309553+0000","last_peered":"2024-05-08T05:50:34.309553+0000","last_clean":"2024-05-08T05:50:34.309553+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:50:34.309553+0000","last_undegraded":"2024-05-08T05:50:34.309553+0000","last_fullsized":"2024-05-08T05:50:34.309553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65925,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":94,"num_write":90,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":186969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":495642,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":1141832,"data_compressed":62105,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070508,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":500320,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266738,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":958464,"data_stored":496246,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462966,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":1090688,"data_compressed":62105,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":491546,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":551270,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":491636,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":551304,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":491650,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":495664,"data_compressed":26462,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":1086192,"data_compressed":62105,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":59758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:51:33.717 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:51:33.718 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T05:51:33.718 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T05:51:33.978 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:34.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:34 smithi003 ceph-mon[30137]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:51:34.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:34 smithi003 ceph-mon[30137]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-05-08T05:51:34.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-08T05:51:34.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:34 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-08T05:51:34.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:34 smithi089 ceph-mon[34495]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:51:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:34 smithi089 ceph-mon[34495]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-05-08T05:51:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-08T05:51:34.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:34 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-08T05:51:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:34 smithi104 ceph-mon[34559]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:51:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:34 smithi104 ceph-mon[34559]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-05-08T05:51:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]: dispatch 2024-05-08T05:51:34.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:34 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-05-08T05:51:35.398 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:35 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-08T05:51:35.399 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:35 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-08T05:51:35.399 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:35 smithi003 ceph-mon[30137]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T05:51:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 -- 172.21.15.3:0/880749940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c1050f0 msgr2=0x7fd90c10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 --2- 172.21.15.3:0/880749940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c1050f0 0x7fd90c10b9a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd90400b0d0 tx=0x7fd90400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 -- 172.21.15.3:0/880749940 shutdown_connections 2024-05-08T05:51:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 --2- 172.21.15.3:0/880749940 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c1050f0 0x7fd90c10b9a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 --2- 172.21.15.3:0/880749940 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c103550 0x7fd90c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.668 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 --2- 172.21.15.3:0/880749940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd90c104750 0x7fd90c104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 -- 172.21.15.3:0/880749940 >> 172.21.15.3:0/880749940 conn(0x7fd90c0feaf0 msgr2=0x7fd90c100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 -- 172.21.15.3:0/880749940 shutdown_connections 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 -- 172.21.15.3:0/880749940 wait complete. 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 Processor -- start 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.666+0000 7fd914161700 1 -- start start 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd90c103550 0x7fd90c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 0x7fd90c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c1050f0 0x7fd90c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c10f070 con 0x7fd90c104750 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c10eef0 con 0x7fd90c1050f0 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c10f1f0 con 0x7fd90c103550 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 0x7fd90c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 0x7fd90c199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49536/0 (socket says 172.21.15.3:49536) 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 -- 172.21.15.3:0/3502337455 learned_addr learned my addr 172.21.15.3:0/3502337455 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd911efd700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd90c103550 0x7fd90c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9126fe700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c1050f0 0x7fd90c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 -- 172.21.15.3:0/3502337455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd90c103550 msgr2=0x7fd90c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd90c103550 0x7fd90c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 -- 172.21.15.3:0/3502337455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c1050f0 msgr2=0x7fd90c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c1050f0 0x7fd90c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 -- 172.21.15.3:0/3502337455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c1a1170 con 0x7fd90c104750 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9126fe700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c1050f0 0x7fd90c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd9116fc700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 0x7fd90c199a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00c9e0 tx=0x7fd8fc00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc007ac0 con 0x7fd90c104750 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8fc007c60 con 0x7fd90c104750 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.667+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c1a1460 con 0x7fd90c104750 2024-05-08T05:51:35.670 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.668+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc018660 con 0x7fd90c104750 2024-05-08T05:51:35.671 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.668+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c1a1a20 con 0x7fd90c104750 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.669+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c0519e0 con 0x7fd90c104750 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.670+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd8fc00f450 con 0x7fd90c104750 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.671+0000 7fd902ffd700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd8f80b29b0 0x7fd8f80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.671+0000 7fd911efd700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd8f80b29b0 0x7fd8f80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.671+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd8fc0e27b0 con 0x7fd90c104750 2024-05-08T05:51:35.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.671+0000 7fd911efd700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd8f80b29b0 0x7fd8f80b4e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd908006fd0 tx=0x7fd908008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:35.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.675+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8fc014030 con 0x7fd90c104750 2024-05-08T05:51:35.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:35 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-08T05:51:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:35 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-08T05:51:35.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:35 smithi089 ceph-mon[34495]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T05:51:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:35 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-05-08T05:51:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:35 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-05-08T05:51:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:35 smithi104 ceph-mon[34559]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T05:51:35.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.981+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd90c02e540 con 0x7fd90c104750 2024-05-08T05:51:35.984 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.984+0000 7fd902ffd700 1 -- 172.21.15.3:0/3502337455 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fd8fc0a9f80 con 0x7fd90c104750 2024-05-08T05:51:35.984 INFO:teuthology.orchestra.run.smithi003.stdout:[client.0] 2024-05-08T05:51:35.984 INFO:teuthology.orchestra.run.smithi003.stdout: key = AQDnEjtmqSSYOhAA1YGJYD0dk9VoHeTR49uvCg== 2024-05-08T05:51:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd8f80b29b0 msgr2=0x7fd8f80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.986 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd8f80b29b0 0x7fd8f80b4e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd908006fd0 tx=0x7fd908008040 comp rx=0 tx=0).stop 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 msgr2=0x7fd90c199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 0x7fd90c199a50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00c9e0 tx=0x7fd8fc00cda0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 shutdown_connections 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd90c1050f0 0x7fd90c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd90c103550 0x7fd90c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd8f80b29b0 0x7fd8f80b4e70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 --2- 172.21.15.3:0/3502337455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd90c104750 0x7fd90c199a50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 >> 172.21.15.3:0/3502337455 conn(0x7fd90c0feaf0 msgr2=0x7fd90c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 shutdown_connections 2024-05-08T05:51:35.987 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:35.986+0000 7fd914161700 1 -- 172.21.15.3:0/3502337455 wait complete. 2024-05-08T05:51:36.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:36 smithi003 ceph-mon[30137]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 818 B/s wr, 2 op/s 2024-05-08T05:51:36.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:36 smithi003 ceph-mon[30137]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T05:51:36.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:36 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3502337455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:51:36.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:36 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3502337455' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:51:36.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:36 smithi089 ceph-mon[34495]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 818 B/s wr, 2 op/s 2024-05-08T05:51:36.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:36 smithi089 ceph-mon[34495]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T05:51:36.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:36 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3502337455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:51:36.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:36 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3502337455' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:51:36.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:36 smithi104 ceph-mon[34559]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 818 B/s wr, 2 op/s 2024-05-08T05:51:36.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:36 smithi104 ceph-mon[34559]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T05:51:36.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:36 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3502337455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:51:36.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:36 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3502337455' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:51:36.803 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:51:36.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T05:51:36.803 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T05:51:36.846 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T05:51:36.847 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T05:51:36.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T05:51:37.140 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:38.566 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:38 smithi003 ceph-mon[30137]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:38.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:38 smithi089 ceph-mon[34495]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:38.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:38 smithi104 ceph-mon[34559]: pgmap v351: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1 op/s 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 -- 172.21.15.3:0/3199333347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 msgr2=0x7fd1d41103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3199333347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d41103f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0099c0 tx=0x7fd1cc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 -- 172.21.15.3:0/3199333347 shutdown_connections 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3199333347 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d41103f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3199333347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d4103780 0x7fd1d4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3199333347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1d4104130 0x7fd1d4108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 -- 172.21.15.3:0/3199333347 >> 172.21.15.3:0/3199333347 conn(0x7fd1d407a610 msgr2=0x7fd1d407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 -- 172.21.15.3:0/3199333347 shutdown_connections 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.970+0000 7fd1d9216700 1 -- 172.21.15.3:0/3199333347 wait complete. 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 Processor -- start 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 -- start start 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d4103780 0x7fd1d419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1d4104130 0x7fd1d419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d4113640 con 0x7fd1d4108870 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41134c0 con 0x7fd1d4103780 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.971+0000 7fd1d9216700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d41137c0 con 0x7fd1d4104130 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49558/0 (socket says 172.21.15.3:49558) 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 -- 172.21.15.3:0/3223099546 learned_addr learned my addr 172.21.15.3:0/3223099546 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d259c700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1d4104130 0x7fd1d419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d2d9d700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d4103780 0x7fd1d419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 -- 172.21.15.3:0/3223099546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1d4104130 msgr2=0x7fd1d419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1d4104130 0x7fd1d419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 -- 172.21.15.3:0/3223099546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d4103780 msgr2=0x7fd1d419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d4103780 0x7fd1d419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 -- 172.21.15.3:0/3223099546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d4199fe0 con 0x7fd1d4108870 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d359e700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d4199800 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0059d0 tx=0x7fd1cc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc027070 con 0x7fd1d4108870 2024-05-08T05:51:38.980 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d419a270 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1cc003fa0 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc022d10 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.972+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d41a64e0 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.973+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d4104c00 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.975+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd1cc02a070 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.975+0000 7fd1cbfff700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1c00b2910 0x7fd1c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.976+0000 7fd1d2d9d700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1c00b2910 0x7fd1c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.976+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd1cc0e3f10 con 0x7fd1d4108870 2024-05-08T05:51:38.981 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.976+0000 7fd1d2d9d700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1c00b2910 0x7fd1c00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40060b0 tx=0x7fd1c400c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:38.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:38.979+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd1cc031020 con 0x7fd1d4108870 2024-05-08T05:51:39.260 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.259+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd1d419ad30 con 0x7fd1d4108870 2024-05-08T05:51:39.265 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.265+0000 7fd1cbfff700 1 -- 172.21.15.3:0/3223099546 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288637 (secure 0 0 0) 0x7fd1cc013070 con 0x7fd1d4108870 2024-05-08T05:51:39.269 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1c00b2910 msgr2=0x7fd1c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1c00b2910 0x7fd1c00b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1c40060b0 tx=0x7fd1c400c040 comp rx=0 tx=0).stop 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 msgr2=0x7fd1d4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d4199800 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0059d0 tx=0x7fd1cc0057c0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 shutdown_connections 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1c00b2910 0x7fd1c00b4dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd1d4108870 0x7fd1d4199800 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd1d4103780 0x7fd1d419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 --2- 172.21.15.3:0/3223099546 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1d4104130 0x7fd1d419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 >> 172.21.15.3:0/3223099546 conn(0x7fd1d407a610 msgr2=0x7fd1d4101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 shutdown_connections 2024-05-08T05:51:39.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:39.268+0000 7fd1d9216700 1 -- 172.21.15.3:0/3223099546 wait complete. 2024-05-08T05:51:39.585 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:39 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3223099546' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:51:39.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:39 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3223099546' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:51:39.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:39 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3223099546' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:51:39.947 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6800","nonce":2793699832},{"type":"v1","addr":"172.21.15.3:6801","nonce":2793699832}]},"active_addr":"172.21.15.3:6801/2793699832","active_change":"2024-05-08T05:42:33.122236+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24118,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.3:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3161157856}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":864931570}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":3921401880}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":1128678241}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.3:0","nonce":2537053940}]}]} 2024-05-08T05:51:39.953 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T05:51:39.953 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T05:51:39.953 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:51:40.209 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:40.585 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:40 smithi003 ceph-mon[30137]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-08T05:51:40.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:40 smithi089 ceph-mon[34495]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-08T05:51:40.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:40 smithi104 ceph-mon[34559]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-08T05:51:42.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 -- 172.21.15.3:0/3020598476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 msgr2=0x7f5e7410f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 --2- 172.21.15.3:0/3020598476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e7410f290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b0d0 tx=0x7f5e6c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 -- 172.21.15.3:0/3020598476 shutdown_connections 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 --2- 172.21.15.3:0/3020598476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e7410f290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 --2- 172.21.15.3:0/3020598476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 0x7f5e74102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 --2- 172.21.15.3:0/3020598476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 0x7f5e74106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 -- 172.21.15.3:0/3020598476 >> 172.21.15.3:0/3020598476 conn(0x7f5e74079350 msgr2=0x7f5e74079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 -- 172.21.15.3:0/3020598476 shutdown_connections 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 -- 172.21.15.3:0/3020598476 wait complete. 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 Processor -- start 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.075+0000 7f5e7b161700 1 -- start start 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e7b161700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 0x7f5e74104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e7b161700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 0x7f5e741045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.078 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e7b161700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e74104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e7b161700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74112580 con 0x7f5e741076c0 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e7b161700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74112400 con 0x7f5e74102fe0 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e7b161700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e74112700 con 0x7f5e74102630 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e74104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e74104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49584/0 (socket says 172.21.15.3:49584) 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 -- 172.21.15.3:0/2938075022 learned_addr learned my addr 172.21.15.3:0/2938075022 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e73fff700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 0x7f5e741045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e78efd700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 0x7f5e74104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 -- 172.21.15.3:0/2938075022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 msgr2=0x7f5e74104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 0x7f5e74104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 -- 172.21.15.3:0/2938075022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 msgr2=0x7f5e741045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.079 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 0x7f5e741045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 -- 172.21.15.3:0/2938075022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e741a8080 con 0x7f5e741076c0 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e73fff700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 0x7f5e741045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e78efd700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 0x7f5e74104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.076+0000 7f5e796fe700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e74104b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0095c0 tx=0x7f5e6c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.077+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c01f070 con 0x7f5e741076c0 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.077+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e741a8310 con 0x7f5e741076c0 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.077+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6c003c00 con 0x7f5e741076c0 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.077+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c01b6a0 con 0x7f5e741076c0 2024-05-08T05:51:42.080 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.077+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e741a8760 con 0x7f5e741076c0 2024-05-08T05:51:42.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.078+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7406a190 con 0x7f5e741076c0 2024-05-08T05:51:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.079+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5e6c022070 con 0x7f5e741076c0 2024-05-08T05:51:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.080+0000 7f5e71ffb700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e640b2910 0x7f5e640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.080+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5e6c0e3a20 con 0x7f5e741076c0 2024-05-08T05:51:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.080+0000 7f5e78efd700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e640b2910 0x7f5e640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:42.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.081+0000 7f5e78efd700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e640b2910 0x7f5e640b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800bcf0 tx=0x7f5e6800b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:42.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.084+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5e6c0ab2e0 con 0x7f5e741076c0 2024-05-08T05:51:42.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.299+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e741a8a80 con 0x7f5e741076c0 2024-05-08T05:51:42.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.301+0000 7f5e71ffb700 1 -- 172.21.15.3:0/2938075022 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24666 (secure 0 0 0) 0x7f5e6c0aabe0 con 0x7f5e741076c0 2024-05-08T05:51:42.301 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:42.302 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:51:35.298219+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e640b2910 msgr2=0x7f5e640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e640b2910 0x7f5e640b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e6800bcf0 tx=0x7f5e6800b2c0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 msgr2=0x7f5e74104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e74104b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0095c0 tx=0x7f5e6c007ad0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 shutdown_connections 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5e640b2910 0x7f5e640b4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5e741076c0 0x7f5e74104b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e74102630 0x7f5e74104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 --2- 172.21.15.3:0/2938075022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5e74102fe0 0x7f5e741045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 >> 172.21.15.3:0/2938075022 conn(0x7f5e74079350 msgr2=0x7f5e741062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 shutdown_connections 2024-05-08T05:51:42.304 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:42.303+0000 7f5e7b161700 1 -- 172.21.15.3:0/2938075022 wait complete. 2024-05-08T05:51:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:42 smithi003 ceph-mon[30137]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-08T05:51:42.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:42 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2938075022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:42.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:42 smithi089 ceph-mon[34495]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-08T05:51:42.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:42 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2938075022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:42 smithi104 ceph-mon[34559]: pgmap v353: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2 op/s 2024-05-08T05:51:42.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:42 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2938075022' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:43.133 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T05:51:43.133 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:51:43.390 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:44.573 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:44 smithi104 ceph-mon[34559]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 572 B/s rd, 1 op/s 2024-05-08T05:51:44.646 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:44 smithi003 ceph-mon[30137]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 572 B/s rd, 1 op/s 2024-05-08T05:51:44.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:44 smithi089 ceph-mon[34495]: pgmap v354: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 572 B/s rd, 1 op/s 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 -- 172.21.15.3:0/2015636693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 msgr2=0x7f9b3810f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2015636693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b3810f2d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b300099c0 tx=0x7f9b30009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 -- 172.21.15.3:0/2015636693 shutdown_connections 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2015636693 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b3810f2d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2015636693 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b38102620 0x7f9b38102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2015636693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38102fd0 0x7f9b38107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.189+0000 7f9b3f179700 1 -- 172.21.15.3:0/2015636693 >> 172.21.15.3:0/2015636693 conn(0x7f9b38079440 msgr2=0x7f9b38079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 -- 172.21.15.3:0/2015636693 shutdown_connections 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 -- 172.21.15.3:0/2015636693 wait complete. 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 Processor -- start 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 -- start start 2024-05-08T05:51:45.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38102620 0x7f9b3819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b38102fd0 0x7f9b3819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b381a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38112610 con 0x7f9b38107700 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38112490 con 0x7f9b38102fd0 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.190+0000 7f9b3f179700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b38112790 con 0x7f9b38102620 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b381a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b381a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58502/0 (socket says 172.21.15.3:58502) 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 -- 172.21.15.3:0/2858699380 learned_addr learned my addr 172.21.15.3:0/2858699380 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b37fff700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b38102fd0 0x7f9b3819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3cf15700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38102620 0x7f9b3819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 -- 172.21.15.3:0/2858699380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38102620 msgr2=0x7f9b3819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38102620 0x7f9b3819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 -- 172.21.15.3:0/2858699380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b38102fd0 msgr2=0x7f9b3819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.193 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b38102fd0 0x7f9b3819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 -- 172.21.15.3:0/2858699380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b381a5f70 con 0x7f9b38107700 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3d716700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b381a5a30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b30005910 tx=0x7f9b30009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30027070 con 0x7f9b38107700 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b30003f20 con 0x7f9b38107700 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b381a61a0 con 0x7f9b38107700 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.191+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30022ba0 con 0x7f9b38107700 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.192+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b381a6650 con 0x7f9b38107700 2024-05-08T05:51:45.194 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.192+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3806a190 con 0x7f9b38107700 2024-05-08T05:51:45.195 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.195+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f9b3002a070 con 0x7f9b38107700 2024-05-08T05:51:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.196+0000 7f9b35ffb700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b240b2990 0x7f9b240b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.196+0000 7f9b3cf15700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b240b2990 0x7f9b240b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.196+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9b300e3490 con 0x7f9b38107700 2024-05-08T05:51:45.196 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.196+0000 7f9b3cf15700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b240b2990 0x7f9b240b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009a10 tx=0x7f9b20009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:45.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.199+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b30013070 con 0x7f9b38107700 2024-05-08T05:51:45.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.414+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b3802e630 con 0x7f9b38107700 2024-05-08T05:51:45.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.416+0000 7f9b35ffb700 1 -- 172.21.15.3:0/2858699380 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24666 (secure 0 0 0) 0x7f9b300aacf0 con 0x7f9b38107700 2024-05-08T05:51:45.417 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:51:45.417 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:51:35.298219+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b240b2990 msgr2=0x7f9b240b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b240b2990 0x7f9b240b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009a10 tx=0x7f9b20009190 comp rx=0 tx=0).stop 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 msgr2=0x7f9b381a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b381a5a30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9b30005910 tx=0x7f9b30009580 comp rx=0 tx=0).stop 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 shutdown_connections 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b240b2990 0x7f9b240b4e50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b38107700 0x7f9b381a5a30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.418+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b38102620 0x7f9b3819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.419+0000 7f9b3f179700 1 --2- 172.21.15.3:0/2858699380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b38102fd0 0x7f9b3819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:45.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.419+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 >> 172.21.15.3:0/2858699380 conn(0x7f9b38079440 msgr2=0x7f9b38100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.419+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 shutdown_connections 2024-05-08T05:51:45.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:45.419+0000 7f9b3f179700 1 -- 172.21.15.3:0/2858699380 wait complete. 2024-05-08T05:51:46.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T05:51:46.009 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T05:51:46.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T05:51:46.010 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T05:51:46.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T05:51:46.011 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T05:51:46.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T05:51:46.012 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T05:51:46.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T05:51:46.013 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T05:51:46.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T05:51:46.014 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T05:51:46.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:46 smithi003 ceph-mon[30137]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-08T05:51:46.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:46 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2858699380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:46.721 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:46.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:46 smithi089 ceph-mon[34495]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-08T05:51:46.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:46 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2858699380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:46 smithi104 ceph-mon[34559]: pgmap v355: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-08T05:51:46.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:46 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2858699380' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:51:46.784 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.164 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.165 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.180 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.181 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:48.582 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:48 smithi003 ceph-mon[30137]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 857 B/s rd, 1 op/s 2024-05-08T05:51:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:48 smithi089 ceph-mon[34495]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 857 B/s rd, 1 op/s 2024-05-08T05:51:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:48 smithi104 ceph-mon[34559]: pgmap v356: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 857 B/s rd, 1 op/s 2024-05-08T05:51:50.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:50 smithi003 ceph-mon[30137]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:51:50.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:50 smithi089 ceph-mon[34495]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:51:50.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:50 smithi104 ceph-mon[34559]: pgmap v357: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:51:51.049 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:51.050 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:51.052 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:51.056 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:51.056 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:51:52.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:52 smithi003 ceph-mon[30137]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:52.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:52 smithi089 ceph-mon[34495]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:52 smithi104 ceph-mon[34559]: pgmap v358: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.821+0000 7f0534783700 1 -- 172.21.15.3:0/4155700281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10adc0 msgr2=0x7f052c1160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.821+0000 7f0534783700 1 --2- 172.21.15.3:0/4155700281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10adc0 0x7f052c1160b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f051c009990 tx=0x7f051c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 -- 172.21.15.3:0/4155700281 shutdown_connections 2024-05-08T05:51:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 --2- 172.21.15.3:0/4155700281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f052c1165f0 0x7f052c1189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 --2- 172.21.15.3:0/4155700281 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c10a4a0 0x7f052c10a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.822 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 --2- 172.21.15.3:0/4155700281 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10adc0 0x7f052c1160b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 -- 172.21.15.3:0/4155700281 >> 172.21.15.3:0/4155700281 conn(0x7f052c06e610 msgr2=0x7f052c06ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 -- 172.21.15.3:0/4155700281 shutdown_connections 2024-05-08T05:51:52.823 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 -- 172.21.15.3:0/4155700281 wait complete. 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.822+0000 7f0534783700 1 Processor -- start 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 -- start start 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10a4a0 0x7f052c1a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f052c10adc0 0x7f052c1a2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c1165f0 0x7f052c1a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f052c11bee0 con 0x7f052c10a4a0 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f052c11bd60 con 0x7f052c1165f0 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0534783700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f052c11c060 con 0x7f052c10adc0 2024-05-08T05:51:52.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10a4a0 0x7f052c1a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10a4a0 0x7f052c1a23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40400/0 (socket says 172.21.15.3:40400) 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 -- 172.21.15.3:0/3085752082 learned_addr learned my addr 172.21.15.3:0/3085752082 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0532d20700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c1165f0 0x7f052c1a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0531d1e700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f052c10adc0 0x7f052c1a2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 -- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f052c10adc0 msgr2=0x7f052c1a2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f052c10adc0 0x7f052c1a2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 -- 172.21.15.3:0/3085752082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c1165f0 msgr2=0x7f052c1a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f052c1165f0 0x7f052c1a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f053251f700 1 -- 172.21.15.3:0/3085752082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052c1a7400 con 0x7f052c10a4a0 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.823+0000 7f0531d1e700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f052c10adc0 0x7f052c1a2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.824+0000 7f053251f700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10a4a0 0x7f052c1a23f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.824+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528016070 con 0x7f052c10a4a0 2024-05-08T05:51:52.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.824+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052c1a76f0 con 0x7f052c10a4a0 2024-05-08T05:51:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.824+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052c10be40 con 0x7f052c10a4a0 2024-05-08T05:51:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.824+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0528009df0 con 0x7f052c10a4a0 2024-05-08T05:51:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.824+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528020060 con 0x7f052c10a4a0 2024-05-08T05:51:52.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.825+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0510001030 con 0x7f052c10a4a0 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.827+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0528019070 con 0x7f052c10a4a0 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.827+0000 7f05237fe700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f05180b2990 0x7f05180b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f0531d1e700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f05180b2990 0x7f05180b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f05280e27c0 con 0x7f052c10a4a0 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f05237fe700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] conn(0x7f05180bdbf0 0x7f05180c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 --> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f052801dec0 con 0x7f05180bdbf0 2024-05-08T05:51:52.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f05280e2c10 con 0x7f052c10a4a0 2024-05-08T05:51:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f0531d1e700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f05180b2990 0x7f05180b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f051c005b40 tx=0x7f051c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.828+0000 7f0532d20700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] conn(0x7f05180bdbf0 0x7f05180c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.829+0000 7f0532d20700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] conn(0x7f05180bdbf0 0x7f05180c0090 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:52.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.830+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== osd.5 v2:172.21.15.104:6808/4290182422 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f052801dec0 con 0x7f05180bdbf0 2024-05-08T05:51:52.847 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.847+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 --> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0510002e60 con 0x7f05180bdbf0 2024-05-08T05:51:52.848 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f05237fe700 1 -- 172.21.15.3:0/3085752082 <== osd.5 v2:172.21.15.104:6808/4290182422 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0510002e60 con 0x7f05180bdbf0 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] conn(0x7f05180bdbf0 msgr2=0x7f05180c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] conn(0x7f05180bdbf0 0x7f05180c0090 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f05180b2990 msgr2=0x7f05180b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f05180b2990 0x7f05180b4e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f051c005b40 tx=0x7f051c01a040 comp rx=0 tx=0).stop 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10a4a0 msgr2=0x7f052c1a23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 --2- 172.21.15.3:0/3085752082 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f052c10a4a0 0x7f052c1a23f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f052800d950 tx=0x7f052800dc60 comp rx=0 tx=0).stop 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0532d20700 1 -- 172.21.15.3:0/3085752082 reap_dead start 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 shutdown_connections 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 >> 172.21.15.3:0/3085752082 conn(0x7f052c06e610 msgr2=0x7f052c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 shutdown_connections 2024-05-08T05:51:52.849 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:52.848+0000 7f0534783700 1 -- 172.21.15.3:0/3085752082 wait complete. 2024-05-08T05:51:53.946 INFO:teuthology.orchestra.run.smithi003.stdout:154618822726 2024-05-08T05:51:53.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T05:51:54.651 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:54 smithi003 ceph-mon[30137]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:54.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:54 smithi089 ceph-mon[34495]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:54.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:54 smithi104 ceph-mon[34559]: pgmap v359: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 -- 172.21.15.3:0/1446160431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac0806c780 msgr2=0x7fac08108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 --2- 172.21.15.3:0/1446160431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac0806c780 0x7fac08108140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fabf80099c0 tx=0x7fabf8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 -- 172.21.15.3:0/1446160431 shutdown_connections 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 --2- 172.21.15.3:0/1446160431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac08108870 0x7fac081103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 --2- 172.21.15.3:0/1446160431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac0806bdd0 0x7fac0806c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 --2- 172.21.15.3:0/1446160431 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac0806c780 0x7fac08108140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 -- 172.21.15.3:0/1446160431 >> 172.21.15.3:0/1446160431 conn(0x7fac08079350 msgr2=0x7fac08079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 -- 172.21.15.3:0/1446160431 shutdown_connections 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.017+0000 7fac10769700 1 -- 172.21.15.3:0/1446160431 wait complete. 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 Processor -- start 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 -- start start 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac0806bdd0 0x7fac081a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac0806c780 0x7fac081a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac08108870 0x7fac0819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113650 con 0x7fac08108870 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fac081134d0 con 0x7fac0806c780 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac10769700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac081137d0 con 0x7fac0806bdd0 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac0ed06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac08108870 0x7fac0819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac0ed06700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac08108870 0x7fac0819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40416/0 (socket says 172.21.15.3:40416) 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.018+0000 7fac0ed06700 1 -- 172.21.15.3:0/1263995849 learned_addr learned my addr 172.21.15.3:0/1263995849 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0dd04700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac0806c780 0x7fac081a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0e505700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac0806bdd0 0x7fac081a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0ed06700 1 -- 172.21.15.3:0/1263995849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac0806bdd0 msgr2=0x7fac081a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0ed06700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac0806bdd0 0x7fac081a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0ed06700 1 -- 172.21.15.3:0/1263995849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac0806c780 msgr2=0x7fac081a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0ed06700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fac0806c780 0x7fac081a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0ed06700 1 -- 172.21.15.3:0/1263995849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0819c2b0 con 0x7fac08108870 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac0ed06700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac08108870 0x7fac0819bb50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac0000ef30 tx=0x7fac0000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac000092e0 con 0x7fac08108870 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac00004bb0 con 0x7fac08108870 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0819c540 con 0x7fac08108870 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac00004020 con 0x7fac08108870 2024-05-08T05:51:56.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.019+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a8810 con 0x7fac08108870 2024-05-08T05:51:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.020+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac0810f240 con 0x7fac08108870 2024-05-08T05:51:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.022+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fac0001b030 con 0x7fac08108870 2024-05-08T05:51:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.023+0000 7fabff7fe700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fabf40b2910 0x7fabf40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.023+0000 7fac0e505700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fabf40b2910 0x7fabf40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.023+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fac000e2b70 con 0x7fac08108870 2024-05-08T05:51:56.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.024+0000 7fabff7fe700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] conn(0x7fabf40bdb70 0x7fabf40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.024+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 --> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac0000be50 con 0x7fabf40bdb70 2024-05-08T05:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.024+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fac000e2fb0 con 0x7fac08108870 2024-05-08T05:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.024+0000 7fac0dd04700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] conn(0x7fabf40bdb70 0x7fabf40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.024+0000 7fac0e505700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fabf40b2910 0x7fabf40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac04005950 tx=0x7fac0400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.024+0000 7fac0dd04700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] conn(0x7fabf40bdb70 0x7fabf40c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:56.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.025+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== osd.6 v2:172.21.15.3:6818/1882383850 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac0000be50 con 0x7fabf40bdb70 2024-05-08T05:51:56.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.043+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 --> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac0806b420 con 0x7fabf40bdb70 2024-05-08T05:51:56.043 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.043+0000 7fabff7fe700 1 -- 172.21.15.3:0/1263995849 <== osd.6 v2:172.21.15.3:6818/1882383850 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac0806b420 con 0x7fabf40bdb70 2024-05-08T05:51:56.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] conn(0x7fabf40bdb70 msgr2=0x7fabf40c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] conn(0x7fabf40bdb70 0x7fabf40c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fabf40b2910 msgr2=0x7fabf40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fabf40b2910 0x7fabf40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac04005950 tx=0x7fac0400b410 comp rx=0 tx=0).stop 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac08108870 msgr2=0x7fac0819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 --2- 172.21.15.3:0/1263995849 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fac08108870 0x7fac0819bb50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac0000ef30 tx=0x7fac0000c3a0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac0ed06700 1 -- 172.21.15.3:0/1263995849 reap_dead start 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 shutdown_connections 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 >> 172.21.15.3:0/1263995849 conn(0x7fac08079350 msgr2=0x7fac0810e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 shutdown_connections 2024-05-08T05:51:56.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.044+0000 7fac10769700 1 -- 172.21.15.3:0/1263995849 wait complete. 2024-05-08T05:51:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 -- 172.21.15.3:0/1762429949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 msgr2=0x7f0a7c110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 --2- 172.21.15.3:0/1762429949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c110340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0a7400b0a0 tx=0x7f0a7400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 -- 172.21.15.3:0/1762429949 shutdown_connections 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 --2- 172.21.15.3:0/1762429949 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c110340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 --2- 172.21.15.3:0/1762429949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c109520 0x7f0a7c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 --2- 172.21.15.3:0/1762429949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a7c101a90 0x7f0a7c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 -- 172.21.15.3:0/1762429949 >> 172.21.15.3:0/1762429949 conn(0x7f0a7c078a10 msgr2=0x7f0a7c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 -- 172.21.15.3:0/1762429949 shutdown_connections 2024-05-08T05:51:56.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.439+0000 7f0a83dca700 1 -- 172.21.15.3:0/1762429949 wait complete. 2024-05-08T05:51:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 Processor -- start 2024-05-08T05:51:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 -- start start 2024-05-08T05:51:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c101a90 0x7f0a7c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a7c109520 0x7f0a7c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c113630 con 0x7f0a7c102450 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1134b0 con 0x7f0a7c109520 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a83dca700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7c1137b0 con 0x7f0a7c101a90 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a81365700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a81b66700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c101a90 0x7f0a7c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a81365700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40432/0 (socket says 172.21.15.3:40432) 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a81b66700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c101a90 0x7f0a7c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:51012/0 (socket says 172.21.15.3:51012) 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a81365700 1 -- 172.21.15.3:0/3612699648 learned_addr learned my addr 172.21.15.3:0/3612699648 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.440+0000 7f0a82367700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a7c109520 0x7f0a7c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81365700 1 -- 172.21.15.3:0/3612699648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c101a90 msgr2=0x7f0a7c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81365700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c101a90 0x7f0a7c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81365700 1 -- 172.21.15.3:0/3612699648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a7c109520 msgr2=0x7f0a7c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81365700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0a7c109520 0x7f0a7c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81365700 1 -- 172.21.15.3:0/3612699648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7c19a070 con 0x7f0a7c102450 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81b66700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a7c101a90 0x7f0a7c19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a81365700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c19fc90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c9b0 tx=0x7f0a6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c007b00 con 0x7f0a7c102450 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a6c007ca0 con 0x7f0a7c102450 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7c19a360 con 0x7f0a7c102450 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c018660 con 0x7f0a7c102450 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.441+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7c1a6620 con 0x7f0a7c102450 2024-05-08T05:51:56.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.443+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a7c063bb0 con 0x7f0a7c102450 2024-05-08T05:51:56.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.445+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f0a6c00f450 con 0x7f0a7c102450 2024-05-08T05:51:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.446+0000 7f0a72ffd700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0a680b2910 0x7f0a680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.446+0000 7f0a81b66700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0a680b2910 0x7f0a680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.446+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0a6c0e2b80 con 0x7f0a7c102450 2024-05-08T05:51:56.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.447+0000 7f0a72ffd700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] conn(0x7f0a680bdb70 0x7f0a680c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:51:56.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.447+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 --> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a6c015e70 con 0x7f0a680bdb70 2024-05-08T05:51:56.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.447+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c0e2fd0 con 0x7f0a7c102450 2024-05-08T05:51:56.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.447+0000 7f0a82367700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] conn(0x7f0a680bdb70 0x7f0a680c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:51:56.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.447+0000 7f0a81b66700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0a680b2910 0x7f0a680b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005e10 tx=0x7f0a78005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:56.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.448+0000 7f0a82367700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] conn(0x7f0a680bdb70 0x7f0a680c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:51:56.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.448+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== osd.3 v2:172.21.15.3:6810/2348325148 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a6c015e70 con 0x7f0a680bdb70 2024-05-08T05:51:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.466+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 --> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0a7c063fb0 con 0x7f0a680bdb70 2024-05-08T05:51:56.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.466+0000 7f0a72ffd700 1 -- 172.21.15.3:0/3612699648 <== osd.3 v2:172.21.15.3:6810/2348325148 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0a7c063fb0 con 0x7f0a680bdb70 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.466+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] conn(0x7f0a680bdb70 msgr2=0x7f0a680c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.466+0000 7f0a83dca700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] conn(0x7f0a680bdb70 0x7f0a680c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0a680b2910 msgr2=0x7f0a680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0a680b2910 0x7f0a680b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0a78005e10 tx=0x7f0a78005d60 comp rx=0 tx=0).stop 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 msgr2=0x7f0a7c19fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 --2- 172.21.15.3:0/3612699648 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0a7c102450 0x7f0a7c19fc90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00c9b0 tx=0x7f0a6c00ccc0 comp rx=0 tx=0).stop 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a82367700 1 -- 172.21.15.3:0/3612699648 reap_dead start 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 shutdown_connections 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 >> 172.21.15.3:0/3612699648 conn(0x7f0a7c078a10 msgr2=0x7f0a7c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:51:56.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 shutdown_connections 2024-05-08T05:51:56.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:56.467+0000 7f0a83dca700 1 -- 172.21.15.3:0/3612699648 wait complete. 2024-05-08T05:51:56.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:56 smithi003 ceph-mon[30137]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:51:56.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:56 smithi089 ceph-mon[34495]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:51:56.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:56 smithi104 ceph-mon[34559]: pgmap v360: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:51:57.605 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:57 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:51:57.659 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:57 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:51:57.678 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:57 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:51:58.409 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:58 smithi003 ceph-mon[30137]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:58.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:58 smithi089 ceph-mon[34495]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:58.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:58 smithi104 ceph-mon[34559]: pgmap v361: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:51:59.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:59 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:59 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:59 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:51:59 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:59 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:59 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:59 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:51:59 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:59 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:59 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:59 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:51:59.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:51:59 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:00.069 INFO:teuthology.orchestra.run.smithi003.stdout:180388626498 2024-05-08T05:52:00.069 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T05:52:00.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:00 smithi003 ceph-mon[30137]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:00 smithi089 ceph-mon[34495]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:00 smithi104 ceph-mon[34559]: pgmap v362: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:01.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 -- 172.21.15.3:0/2425223616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec075ff0 msgr2=0x7f5dec10f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 --2- 172.21.15.3:0/2425223616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec075ff0 0x7f5dec10f9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009990 tx=0x7f5dd8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 -- 172.21.15.3:0/2425223616 shutdown_connections 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 --2- 172.21.15.3:0/2425223616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec10ff00 0x7f5dec1122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 --2- 172.21.15.3:0/2425223616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 0x7f5dec075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 --2- 172.21.15.3:0/2425223616 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec075ff0 0x7f5dec10f9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 -- 172.21.15.3:0/2425223616 >> 172.21.15.3:0/2425223616 conn(0x7f5dec07a530 msgr2=0x7f5dec07a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 -- 172.21.15.3:0/2425223616 shutdown_connections 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.677+0000 7f5df30c8700 1 -- 172.21.15.3:0/2425223616 wait complete. 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 Processor -- start 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 -- start start 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 0x7f5dec1a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec075ff0 0x7f5dec1a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec10ff00 0x7f5dec19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec115860 con 0x7f5dec10ff00 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1156e0 con 0x7f5dec0756d0 2024-05-08T05:52:01.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.678+0000 7f5df30c8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1159e0 con 0x7f5dec075ff0 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec10ff00 0x7f5dec19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df0e64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 0x7f5dec1a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df0e64700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 0x7f5dec1a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:46452/0 (socket says 172.21.15.3:46452) 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec10ff00 0x7f5dec19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40452/0 (socket says 172.21.15.3:40452) 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df0e64700 1 -- 172.21.15.3:0/443735231 learned_addr learned my addr 172.21.15.3:0/443735231 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5debfff700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec075ff0 0x7f5dec1a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 -- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec075ff0 msgr2=0x7f5dec1a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5dec075ff0 0x7f5dec1a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 -- 172.21.15.3:0/443735231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 msgr2=0x7f5dec1a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 0x7f5dec1a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 -- 172.21.15.3:0/443735231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19c400 con 0x7f5dec10ff00 2024-05-08T05:52:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df0e64700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5dec0756d0 0x7f5dec1a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:01.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df1665700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec10ff00 0x7f5dec19bb20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5de4007f10 tx=0x7f5de400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4004db0 con 0x7f5dec10ff00 2024-05-08T05:52:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5de4009e20 con 0x7f5dec10ff00 2024-05-08T05:52:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19c6f0 con 0x7f5dec10ff00 2024-05-08T05:52:01.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.679+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4015400 con 0x7f5dec10ff00 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.680+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a89d0 con 0x7f5dec10ff00 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.680+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5dec063c30 con 0x7f5dec10ff00 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.682+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f5de4005260 con 0x7f5dec10ff00 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.682+0000 7f5de9ffb700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5ddc0b2990 0x7f5ddc0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5df0e64700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5ddc0b2990 0x7f5ddc0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f5de40e2510 con 0x7f5dec10ff00 2024-05-08T05:52:01.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5de9ffb700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] conn(0x7f5ddc0bdbf0 0x7f5ddc0c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 --> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5de400ee50 con 0x7f5ddc0bdbf0 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f5de40e2960 con 0x7f5dec10ff00 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5df0e64700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5ddc0b2990 0x7f5ddc0b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5de00104e0 tx=0x7f5de0007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.683+0000 7f5debfff700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] conn(0x7f5ddc0bdbf0 0x7f5ddc0c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.684+0000 7f5debfff700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] conn(0x7f5ddc0bdbf0 0x7f5ddc0c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.684+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== osd.2 v2:172.21.15.104:6800/3444157225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5de400ee50 con 0x7f5ddc0bdbf0 2024-05-08T05:52:01.210 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.701+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 --> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5dec0640e0 con 0x7f5ddc0bdbf0 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5de9ffb700 1 -- 172.21.15.3:0/443735231 <== osd.2 v2:172.21.15.104:6800/3444157225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5dec0640e0 con 0x7f5ddc0bdbf0 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] conn(0x7f5ddc0bdbf0 msgr2=0x7f5ddc0c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] conn(0x7f5ddc0bdbf0 0x7f5ddc0c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5ddc0b2990 msgr2=0x7f5ddc0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f5ddc0b2990 0x7f5ddc0b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5de00104e0 tx=0x7f5de0007480 comp rx=0 tx=0).stop 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec10ff00 msgr2=0x7f5dec19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 --2- 172.21.15.3:0/443735231 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5dec10ff00 0x7f5dec19bb20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5de4007f10 tx=0x7f5de400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df1665700 1 -- 172.21.15.3:0/443735231 reap_dead start 2024-05-08T05:52:01.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 shutdown_connections 2024-05-08T05:52:01.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 >> 172.21.15.3:0/443735231 conn(0x7f5dec07a530 msgr2=0x7f5dec100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:01.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 shutdown_connections 2024-05-08T05:52:01.212 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:51:58.702+0000 7f5df30c8700 1 -- 172.21.15.3:0/443735231 wait complete. 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 -- 172.21.15.3:0/2045254527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103ef0 msgr2=0x7fe4601103a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 --2- 172.21.15.3:0/2045254527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103ef0 0x7fe4601103a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4580099c0 tx=0x7fe458009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 -- 172.21.15.3:0/2045254527 shutdown_connections 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 --2- 172.21.15.3:0/2045254527 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103ef0 0x7fe4601103a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 --2- 172.21.15.3:0/2045254527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4601095d0 0x7fe4601099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 --2- 172.21.15.3:0/2045254527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe460103530 0x7fe4601039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 -- 172.21.15.3:0/2045254527 >> 172.21.15.3:0/2045254527 conn(0x7fe46007a690 msgr2=0x7fe46007aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 -- 172.21.15.3:0/2045254527 shutdown_connections 2024-05-08T05:52:01.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.614+0000 7fe46844d700 1 -- 172.21.15.3:0/2045254527 wait complete. 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 Processor -- start 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 -- start start 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103530 0x7fe4600757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe460103ef0 0x7fe460075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4601095d0 0x7fe460076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe4601136e0 con 0x7fe460103530 2024-05-08T05:52:01.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe460113560 con 0x7fe4601095d0 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.615+0000 7fe46844d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe460113860 con 0x7fe460103ef0 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103530 0x7fe4600757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103530 0x7fe4600757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40482/0 (socket says 172.21.15.3:40482) 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 -- 172.21.15.3:0/2767316293 learned_addr learned my addr 172.21.15.3:0/2767316293 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4669ea700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4601095d0 0x7fe460076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4659e8700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe460103ef0 0x7fe460075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 -- 172.21.15.3:0/2767316293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe460103ef0 msgr2=0x7fe460075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe460103ef0 0x7fe460075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 -- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4601095d0 msgr2=0x7fe460076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe4601095d0 0x7fe460076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 -- 172.21.15.3:0/2767316293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4601a7ff0 con 0x7fe460103530 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe4661e9700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103530 0x7fe4600757a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe450007f10 tx=0x7fe45000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45000dbd0 con 0x7fe460103530 2024-05-08T05:52:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4601a82e0 con 0x7fe460103530 2024-05-08T05:52:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe45000dd70 con 0x7fe460103530 2024-05-08T05:52:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe450005a10 con 0x7fe460103530 2024-05-08T05:52:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.616+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601a87d0 con 0x7fe460103530 2024-05-08T05:52:01.619 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.617+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe460063bb0 con 0x7fe460103530 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.619+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fe45001a020 con 0x7fe460103530 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.619+0000 7fe45f7fe700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe4480b2910 0x7fe4480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe4659e8700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe4480b2910 0x7fe4480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe4500e2550 con 0x7fe460103530 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe45f7fe700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] conn(0x7fe4480bdb70 0x7fe4480c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 --> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4480c06e0 con 0x7fe4480bdb70 2024-05-08T05:52:01.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fe4500e29a0 con 0x7fe460103530 2024-05-08T05:52:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe4669ea700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] conn(0x7fe4480bdb70 0x7fe4480c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe4659e8700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe4480b2910 0x7fe4480b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe454005950 tx=0x7fe4540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.620+0000 7fe4669ea700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] conn(0x7fe4480bdb70 0x7fe4480c0010 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:01.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.621+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== osd.7 v2:172.21.15.89:6816/2376892451 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4480c06e0 con 0x7fe4480bdb70 2024-05-08T05:52:01.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.638+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 --> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe460064030 con 0x7fe4480bdb70 2024-05-08T05:52:01.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe45f7fe700 1 -- 172.21.15.3:0/2767316293 <== osd.7 v2:172.21.15.89:6816/2376892451 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe460064030 con 0x7fe4480bdb70 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] conn(0x7fe4480bdb70 msgr2=0x7fe4480c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] conn(0x7fe4480bdb70 0x7fe4480c0010 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe4480b2910 msgr2=0x7fe4480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe4480b2910 0x7fe4480b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe454005950 tx=0x7fe4540058e0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103530 msgr2=0x7fe4600757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 --2- 172.21.15.3:0/2767316293 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe460103530 0x7fe4600757a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe450007f10 tx=0x7fe45000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe4669ea700 1 -- 172.21.15.3:0/2767316293 reap_dead start 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 shutdown_connections 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 >> 172.21.15.3:0/2767316293 conn(0x7fe46007a690 msgr2=0x7fe460102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 shutdown_connections 2024-05-08T05:52:01.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:01.639+0000 7fe46844d700 1 -- 172.21.15.3:0/2767316293 wait complete. 2024-05-08T05:52:02.855 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:02 smithi003 ceph-mon[30137]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:03.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:02 smithi089 ceph-mon[34495]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:02 smithi104 ceph-mon[34559]: pgmap v363: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:04.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 -- 172.21.15.3:0/2744600476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544810fef0 msgr2=0x7f54481122e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:04.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 --2- 172.21.15.3:0/2744600476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544810fef0 0x7f54481122e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f544000b0a0 tx=0x7f544000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 -- 172.21.15.3:0/2744600476 shutdown_connections 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 --2- 172.21.15.3:0/2744600476 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544810fef0 0x7f54481122e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 --2- 172.21.15.3:0/2744600476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f544806bc50 0x7f544806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 --2- 172.21.15.3:0/2744600476 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f544806c570 0x7f544810f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 -- 172.21.15.3:0/2744600476 >> 172.21.15.3:0/2744600476 conn(0x7f5448079440 msgr2=0x7f5448079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 -- 172.21.15.3:0/2744600476 shutdown_connections 2024-05-08T05:52:04.235 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.234+0000 7f544f397700 1 -- 172.21.15.3:0/2744600476 wait complete. 2024-05-08T05:52:04.236 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 Processor -- start 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 -- start start 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544806bc50 0x7f54481a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f544806c570 0x7f54481a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f544810fef0 0x7f544819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5448115890 con 0x7f544806bc50 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5448115710 con 0x7f544806c570 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.235+0000 7f544f397700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5448115a10 con 0x7f544810fef0 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544806bc50 0x7f54481a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544806bc50 0x7f54481a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34020/0 (socket says 172.21.15.3:34020) 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 -- 172.21.15.3:0/1354078062 learned_addr learned my addr 172.21.15.3:0/1354078062 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544c932700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f544806c570 0x7f54481a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:04.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d934700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f544810fef0 0x7f544819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 -- 172.21.15.3:0/1354078062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f544810fef0 msgr2=0x7f544819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f544810fef0 0x7f544819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 -- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f544806c570 msgr2=0x7f54481a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f544806c570 0x7f54481a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 -- 172.21.15.3:0/1354078062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f544819c450 con 0x7f544806bc50 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544d133700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544806bc50 0x7f54481a1a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f544400c9b0 tx=0x7f544400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444007b00 con 0x7f544806bc50 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f544400ce50 con 0x7f544806bc50 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.236+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f544819c740 con 0x7f544806bc50 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.237+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444018660 con 0x7f544806bc50 2024-05-08T05:52:04.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.237+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481a89d0 con 0x7f544806bc50 2024-05-08T05:52:04.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.238+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5448063bf0 con 0x7f544806bc50 2024-05-08T05:52:04.239 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.239+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f544400f450 con 0x7f544806bc50 2024-05-08T05:52:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f543e7fc700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f54340b2910 0x7f54340b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f544c932700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f54340b2910 0x7f54340b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:04.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f54440e2660 con 0x7f544806bc50 2024-05-08T05:52:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f543e7fc700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] conn(0x7f54340bdb70 0x7f54340c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 --> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5444012ec0 con 0x7f54340bdb70 2024-05-08T05:52:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f54440e9050 con 0x7f544806bc50 2024-05-08T05:52:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f544c932700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f54340b2910 0x7f54340b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f544819d270 tx=0x7f543800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.240+0000 7f544d934700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] conn(0x7f54340bdb70 0x7f54340c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:04.241 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.241+0000 7f544d934700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] conn(0x7f54340bdb70 0x7f54340c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:04.242 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.242+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== osd.4 v2:172.21.15.89:6808/3658777632 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5444012ec0 con 0x7f54340bdb70 2024-05-08T05:52:04.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.262+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 --> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5448063ff0 con 0x7f54340bdb70 2024-05-08T05:52:04.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.262+0000 7f543e7fc700 1 -- 172.21.15.3:0/1354078062 <== osd.4 v2:172.21.15.89:6808/3658777632 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5448063ff0 con 0x7f54340bdb70 2024-05-08T05:52:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] conn(0x7f54340bdb70 msgr2=0x7f54340c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] conn(0x7f54340bdb70 0x7f54340c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f54340b2910 msgr2=0x7f54340b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f54340b2910 0x7f54340b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f544819d270 tx=0x7f543800b410 comp rx=0 tx=0).stop 2024-05-08T05:52:04.263 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544806bc50 msgr2=0x7f54481a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 --2- 172.21.15.3:0/1354078062 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f544806bc50 0x7f54481a1a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f544400c9b0 tx=0x7f544400ccc0 comp rx=0 tx=0).stop 2024-05-08T05:52:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544d934700 1 -- 172.21.15.3:0/1354078062 reap_dead start 2024-05-08T05:52:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 shutdown_connections 2024-05-08T05:52:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 >> 172.21.15.3:0/1354078062 conn(0x7f5448079440 msgr2=0x7f5448111340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 shutdown_connections 2024-05-08T05:52:04.264 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:04.263+0000 7f544f397700 1 -- 172.21.15.3:0/1354078062 wait complete. 2024-05-08T05:52:04.661 INFO:teuthology.orchestra.run.smithi003.stdout:107374182481 2024-05-08T05:52:04.662 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T05:52:04.895 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:04 smithi003 ceph-mon[30137]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:05.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:04 smithi089 ceph-mon[34495]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:04 smithi104 ceph-mon[34559]: pgmap v364: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 -- 172.21.15.3:0/1656626475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 msgr2=0x7fc27c06c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1656626475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c06c0f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0099c0 tx=0x7fc26c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 -- 172.21.15.3:0/1656626475 shutdown_connections 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1656626475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc27c06c630 0x7fc27c110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1656626475 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc27c109590 0x7fc27c109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1656626475 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c06c0f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 -- 172.21.15.3:0/1656626475 >> 172.21.15.3:0/1656626475 conn(0x7fc27c079440 msgr2=0x7fc27c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 -- 172.21.15.3:0/1656626475 shutdown_connections 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.070+0000 7fc2838d7700 1 -- 172.21.15.3:0/1656626475 wait complete. 2024-05-08T05:52:06.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 Processor -- start 2024-05-08T05:52:06.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 -- start start 2024-05-08T05:52:06.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc27c06c630 0x7fc27c19c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc27c109590 0x7fc27c1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1136f0 con 0x7fc27c06bc70 2024-05-08T05:52:06.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c113570 con 0x7fc27c06c630 2024-05-08T05:52:06.073 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc2838d7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c113870 con 0x7fc27c109590 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc281673700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc281673700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c19bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34050/0 (socket says 172.21.15.3:34050) 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc281673700 1 -- 172.21.15.3:0/1858787170 learned_addr learned my addr 172.21.15.3:0/1858787170 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.071+0000 7fc280e72700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc27c06c630 0x7fc27c19c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281e74700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc27c109590 0x7fc27c1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281673700 1 -- 172.21.15.3:0/1858787170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc27c109590 msgr2=0x7fc27c1a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281673700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc27c109590 0x7fc27c1a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281673700 1 -- 172.21.15.3:0/1858787170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc27c06c630 msgr2=0x7fc27c19c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281673700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc27c06c630 0x7fc27c19c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281673700 1 -- 172.21.15.3:0/1858787170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c1a0d30 con 0x7fc27c06bc70 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc280e72700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc27c06c630 0x7fc27c19c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc281673700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c19bd20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc278007f10 tx=0x7fc27800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:06.074 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27800dc90 con 0x7fc27c06bc70 2024-05-08T05:52:06.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c1a1020 con 0x7fc27c06bc70 2024-05-08T05:52:06.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc27800de30 con 0x7fc27c06bc70 2024-05-08T05:52:06.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780133f0 con 0x7fc27c06bc70 2024-05-08T05:52:06.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.072+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c1a88a0 con 0x7fc27c06bc70 2024-05-08T05:52:06.075 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.073+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc27c10f110 con 0x7fc27c06bc70 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.074+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fc27801a020 con 0x7fc27c06bc70 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.075+0000 7fc2727fc700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc2680b2910 0x7fc2680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.075+0000 7fc280e72700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc2680b2910 0x7fc2680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.075+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc2780e26c0 con 0x7fc27c06bc70 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.075+0000 7fc2727fc700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] conn(0x7fc2680bdb70 0x7fc2680c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.076+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 --> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2680c06e0 con 0x7fc2680bdb70 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.076+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc2780e2b10 con 0x7fc27c06bc70 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.076+0000 7fc281e74700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] conn(0x7fc2680bdb70 0x7fc2680c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.076+0000 7fc280e72700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc2680b2910 0x7fc2680b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc27c104340 tx=0x7fc26c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:06.076 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.076+0000 7fc281e74700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] conn(0x7fc2680bdb70 0x7fc2680c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:06.077 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.077+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== osd.9 v2:172.21.15.3:6826/1455470831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2680c06e0 con 0x7fc2680bdb70 2024-05-08T05:52:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 --> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc27c104c00 con 0x7fc2680bdb70 2024-05-08T05:52:06.095 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2727fc700 1 -- 172.21.15.3:0/1858787170 <== osd.9 v2:172.21.15.3:6826/1455470831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc27c104c00 con 0x7fc2680bdb70 2024-05-08T05:52:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] conn(0x7fc2680bdb70 msgr2=0x7fc2680c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] conn(0x7fc2680bdb70 0x7fc2680c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc2680b2910 msgr2=0x7fc2680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc2680b2910 0x7fc2680b4dd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc27c104340 tx=0x7fc26c005fb0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.096 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 msgr2=0x7fc27c19bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc2838d7700 1 --2- 172.21.15.3:0/1858787170 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc27c06bc70 0x7fc27c19bd20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc278007f10 tx=0x7fc27800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.095+0000 7fc281e74700 1 -- 172.21.15.3:0/1858787170 reap_dead start 2024-05-08T05:52:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.096+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 shutdown_connections 2024-05-08T05:52:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.096+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 >> 172.21.15.3:0/1858787170 conn(0x7fc27c079440 msgr2=0x7fc27c10e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.096+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 shutdown_connections 2024-05-08T05:52:06.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:06.096+0000 7fc2838d7700 1 -- 172.21.15.3:0/1858787170 wait complete. 2024-05-08T05:52:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 -- 172.21.15.3:0/270207801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974102390 msgr2=0x7ff974102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/270207801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974102390 0x7ff974102810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff964009990 tx=0x7ff964009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 -- 172.21.15.3:0/270207801 shutdown_connections 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/270207801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff974102d50 0x7ff97410f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/270207801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff974108400 0x7ff9741087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/270207801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974102390 0x7ff974102810 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 -- 172.21.15.3:0/270207801 >> 172.21.15.3:0/270207801 conn(0x7ff974079310 msgr2=0x7ff974079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 -- 172.21.15.3:0/270207801 shutdown_connections 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 -- 172.21.15.3:0/270207801 wait complete. 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.968+0000 7ff97b5ed700 1 Processor -- start 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 -- start start 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff974102390 0x7ff974104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff974102d50 0x7ff9741045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974108400 0x7ff974104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff974112520 con 0x7ff974108400 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741123a0 con 0x7ff974102d50 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff97b5ed700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9741126a0 con 0x7ff974102390 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974108400 0x7ff974104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974108400 0x7ff974104af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34036/0 (socket says 172.21.15.3:34036) 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 -- 172.21.15.3:0/2592540735 learned_addr learned my addr 172.21.15.3:0/2592540735 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff978b88700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff974102d50 0x7ff9741045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979389700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff974102390 0x7ff974104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 -- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff974102390 msgr2=0x7ff974104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff974102390 0x7ff974104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 -- 172.21.15.3:0/2592540735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff974102d50 msgr2=0x7ff9741045b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff974102d50 0x7ff9741045b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 -- 172.21.15.3:0/2592540735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9741a8000 con 0x7ff974108400 2024-05-08T05:52:06.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979389700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff974102390 0x7ff974104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.969+0000 7ff979b8a700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974108400 0x7ff974104af0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff96c007d70 tx=0x7ff96c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.970+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c016020 con 0x7ff974108400 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.970+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff96c00ad40 con 0x7ff974108400 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.970+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9741a82f0 con 0x7ff974108400 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.970+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0115e0 con 0x7ff974108400 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.970+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9741a87b0 con 0x7ff974108400 2024-05-08T05:52:06.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.970+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff974106480 con 0x7ff974108400 2024-05-08T05:52:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.972+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff96c019020 con 0x7ff974108400 2024-05-08T05:52:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.973+0000 7ff96a7fc700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff9600b2910 0x7ff9600b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.973+0000 7ff979389700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff9600b2910 0x7ff9600b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.973+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff96c0e2c60 con 0x7ff974108400 2024-05-08T05:52:06.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.973+0000 7ff96a7fc700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] conn(0x7ff9600bdb70 0x7ff9600c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.973+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 --> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff96c01ce50 con 0x7ff9600bdb70 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.973+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff96c0e30b0 con 0x7ff974108400 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.974+0000 7ff979389700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff9600b2910 0x7ff9600b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff970005fd0 tx=0x7ff970005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.974+0000 7ff978b88700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] conn(0x7ff9600bdb70 0x7ff9600c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.974+0000 7ff978b88700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] conn(0x7ff9600bdb70 0x7ff9600c0010 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.975+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== osd.11 v2:172.21.15.104:6824/1990686084 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff96c01ce50 con 0x7ff9600bdb70 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.992+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 --> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff974103ab0 con 0x7ff9600bdb70 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff96a7fc700 1 -- 172.21.15.3:0/2592540735 <== osd.11 v2:172.21.15.104:6824/1990686084 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff974103ab0 con 0x7ff9600bdb70 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] conn(0x7ff9600bdb70 msgr2=0x7ff9600c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] conn(0x7ff9600bdb70 0x7ff9600c0010 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff9600b2910 msgr2=0x7ff9600b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff9600b2910 0x7ff9600b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff970005fd0 tx=0x7ff970005db0 comp rx=0 tx=0).stop 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974108400 msgr2=0x7ff974104af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:06.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 --2- 172.21.15.3:0/2592540735 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff974108400 0x7ff974104af0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff96c007d70 tx=0x7ff96c00a340 comp rx=0 tx=0).stop 2024-05-08T05:52:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff979b8a700 1 -- 172.21.15.3:0/2592540735 reap_dead start 2024-05-08T05:52:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 shutdown_connections 2024-05-08T05:52:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 >> 172.21.15.3:0/2592540735 conn(0x7ff974079310 msgr2=0x7ff9741016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 shutdown_connections 2024-05-08T05:52:06.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:05.993+0000 7ff97b5ed700 1 -- 172.21.15.3:0/2592540735 wait complete. 2024-05-08T05:52:06.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:06 smithi003 ceph-mon[30137]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:07.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:06 smithi089 ceph-mon[34495]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:06 smithi104 ceph-mon[34559]: pgmap v365: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:07.928 INFO:teuthology.orchestra.run.smithi003.stdout:300647710761 2024-05-08T05:52:07.928 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T05:52:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 -- 172.21.15.3:0/3646082664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680103750 msgr2=0x7f4680103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 --2- 172.21.15.3:0/3646082664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680103750 0x7f4680103b30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f46700099c0 tx=0x7f4670009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 -- 172.21.15.3:0/3646082664 shutdown_connections 2024-05-08T05:52:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 --2- 172.21.15.3:0/3646082664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4680108850 0x7f46801103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 --2- 172.21.15.3:0/3646082664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680103750 0x7f4680103b30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 --2- 172.21.15.3:0/3646082664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4680104100 0x7f4680108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 -- 172.21.15.3:0/3646082664 >> 172.21.15.3:0/3646082664 conn(0x7f468007a600 msgr2=0x7f468007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 -- 172.21.15.3:0/3646082664 shutdown_connections 2024-05-08T05:52:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.351+0000 7f4687665700 1 -- 172.21.15.3:0/3646082664 wait complete. 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 Processor -- start 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 -- start start 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4680103750 0x7f468010d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680104100 0x7f468010db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4680108850 0x7f468010e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f46800772b0 con 0x7f4680104100 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4680077130 con 0x7f4680108850 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4687665700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4680077430 con 0x7f4680103750 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4684c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680104100 0x7f468010db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:08.354 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4684c00700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680104100 0x7f468010db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34086/0 (socket says 172.21.15.3:34086) 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4684c00700 1 -- 172.21.15.3:0/3372052772 learned_addr learned my addr 172.21.15.3:0/3372052772 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4685c02700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4680108850 0x7f468010e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4685401700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4680103750 0x7f468010d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4685c02700 1 -- 172.21.15.3:0/3372052772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4680103750 msgr2=0x7f468010d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.352+0000 7f4685c02700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4680103750 0x7f468010d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4685c02700 1 -- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680104100 msgr2=0x7f468010db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4685c02700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680104100 0x7f468010db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4685c02700 1 -- 172.21.15.3:0/3372052772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46801ac3f0 con 0x7f4680108850 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4684c00700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4680104100 0x7f468010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4685c02700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4680108850 0x7f468010e060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f467800efc0 tx=0x7f467800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46780092e0 con 0x7f4680108850 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46801ac680 con 0x7f4680108850 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4678004bb0 con 0x7f4680108850 2024-05-08T05:52:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678004020 con 0x7f4680108850 2024-05-08T05:52:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.353+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46801acb70 con 0x7f4680108850 2024-05-08T05:52:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.354+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4680063bf0 con 0x7f4680108850 2024-05-08T05:52:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.356+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f467801b030 con 0x7f4680108850 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f467e7fc700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46680b2990 0x7f46680b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f4685401700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46680b2990 0x7f46680b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f46780e2c20 con 0x7f4680108850 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f467e7fc700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] conn(0x7f46680bdbf0 0x7f46680c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 --> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4678011e50 con 0x7f46680bdbf0 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f46780e3010 con 0x7f4680108850 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f4685401700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46680b2990 0x7f46680b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4670000c00 tx=0x7f4670005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.357+0000 7f4684c00700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] conn(0x7f46680bdbf0 0x7f46680c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:08.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.358+0000 7f4684c00700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] conn(0x7f46680bdbf0 0x7f46680c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:08.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.359+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== osd.1 v2:172.21.15.89:6800/662806549 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4678011e50 con 0x7f46680bdbf0 2024-05-08T05:52:08.375 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.375+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 --> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4680064070 con 0x7f46680bdbf0 2024-05-08T05:52:08.376 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.376+0000 7f467e7fc700 1 -- 172.21.15.3:0/3372052772 <== osd.1 v2:172.21.15.89:6800/662806549 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4680064070 con 0x7f46680bdbf0 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.376+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] conn(0x7f46680bdbf0 msgr2=0x7f46680c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.376+0000 7f4687665700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] conn(0x7f46680bdbf0 0x7f46680c0090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.376+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46680b2990 msgr2=0x7f46680b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.376+0000 7f4687665700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f46680b2990 0x7f46680b4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4670000c00 tx=0x7f4670005fb0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4680108850 msgr2=0x7f468010e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4687665700 1 --2- 172.21.15.3:0/3372052772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4680108850 0x7f468010e060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f467800efc0 tx=0x7f467800c3a0 comp rx=0 tx=0).stop 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4685c02700 1 -- 172.21.15.3:0/3372052772 reap_dead start 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 shutdown_connections 2024-05-08T05:52:08.377 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 >> 172.21.15.3:0/3372052772 conn(0x7f468007a600 msgr2=0x7f4680101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 shutdown_connections 2024-05-08T05:52:08.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:08.377+0000 7f4687665700 1 -- 172.21.15.3:0/3372052772 wait complete. 2024-05-08T05:52:08.871 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:08 smithi003 ceph-mon[30137]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:09.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:08 smithi089 ceph-mon[34495]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:09.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:08 smithi104 ceph-mon[34559]: pgmap v366: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:09.188 INFO:teuthology.orchestra.run.smithi003.stdout:55834574940 2024-05-08T05:52:09.189 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T05:52:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 -- 172.21.15.3:0/872159770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 msgr2=0x7f7424110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 --2- 172.21.15.3:0/872159770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f7424110290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f741c00b0a0 tx=0x7f741c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 -- 172.21.15.3:0/872159770 shutdown_connections 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 --2- 172.21.15.3:0/872159770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f7424110290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 --2- 172.21.15.3:0/872159770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7424108580 0x7f7424108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 --2- 172.21.15.3:0/872159770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7424101e60 0x7f74241022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 -- 172.21.15.3:0/872159770 >> 172.21.15.3:0/872159770 conn(0x7f74240776c0 msgr2=0x7f7424077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 -- 172.21.15.3:0/872159770 shutdown_connections 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.208+0000 7f742c677700 1 -- 172.21.15.3:0/872159770 wait complete. 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 Processor -- start 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 -- start start 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7424101e60 0x7f742419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f742419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7424108580 0x7f7424199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f74241136c0 con 0x7f7424102820 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7424113540 con 0x7f7424108580 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.209+0000 7f742c677700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7424113840 con 0x7f7424101e60 2024-05-08T05:52:09.651 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f742419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f742419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34068/0 (socket says 172.21.15.3:34068) 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 -- 172.21.15.3:0/463745175 learned_addr learned my addr 172.21.15.3:0/463745175 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f742a413700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7424101e60 0x7f742419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f742ac14700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7424108580 0x7f7424199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 -- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7424101e60 msgr2=0x7f742419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7424101e60 0x7f742419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 -- 172.21.15.3:0/463745175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7424108580 msgr2=0x7f7424199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7424108580 0x7f7424199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.652 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 -- 172.21.15.3:0/463745175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742419a0c0 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f742ac14700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7424108580 0x7f7424199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f7429c12700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f742419fc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f741800e9a0 tx=0x7f741800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741800cbd0 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f741800cd70 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742419a3b0 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418018680 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.210+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241a6620 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.211+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7424063c30 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.213+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f7418010970 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.213+0000 7f74177fe700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74100b2910 0x7f74100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f742a413700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74100b2910 0x7f74100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f74180e3320 con 0x7f7424102820 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f74177fe700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] conn(0x7f74100bdb70 0x7f74100c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 --> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74100c06e0 con 0x7f74100bdb70 2024-05-08T05:52:09.653 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f74180e36d0 con 0x7f7424102820 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f742a413700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74100b2910 0x7f74100b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f74241004e0 tx=0x7f7420009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.214+0000 7f742ac14700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] conn(0x7f74100bdb70 0x7f74100c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.215+0000 7f742ac14700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] conn(0x7f74100bdb70 0x7f74100c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.215+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== osd.8 v2:172.21.15.104:6816/1691421157 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74100c06e0 con 0x7f74100bdb70 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.232+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 --> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f74240640b0 con 0x7f74100bdb70 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f74177fe700 1 -- 172.21.15.3:0/463745175 <== osd.8 v2:172.21.15.104:6816/1691421157 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f74240640b0 con 0x7f74100bdb70 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] conn(0x7f74100bdb70 msgr2=0x7f74100c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] conn(0x7f74100bdb70 0x7f74100c0010 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74100b2910 msgr2=0x7f74100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74100b2910 0x7f74100b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f74241004e0 tx=0x7f7420009260 comp rx=0 tx=0).stop 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 msgr2=0x7f742419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 --2- 172.21.15.3:0/463745175 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7424102820 0x7f742419fc90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f741800e9a0 tx=0x7f741800ed60 comp rx=0 tx=0).stop 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742ac14700 1 -- 172.21.15.3:0/463745175 reap_dead start 2024-05-08T05:52:09.654 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 shutdown_connections 2024-05-08T05:52:09.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 >> 172.21.15.3:0/463745175 conn(0x7f74240776c0 msgr2=0x7f7424100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:09.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 shutdown_connections 2024-05-08T05:52:09.655 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:07.233+0000 7f742c677700 1 -- 172.21.15.3:0/463745175 wait complete. 2024-05-08T05:52:10.083 INFO:teuthology.orchestra.run.smithi003.stdout:253403070516 2024-05-08T05:52:10.084 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T05:52:10.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 -- 172.21.15.3:0/2606898550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 msgr2=0x7fd154106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:10.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 --2- 172.21.15.3:0/2606898550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd154106fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd13c0099c0 tx=0x7fd13c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 -- 172.21.15.3:0/2606898550 shutdown_connections 2024-05-08T05:52:10.175 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 --2- 172.21.15.3:0/2606898550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1541076c0 0x7fd15410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 --2- 172.21.15.3:0/2606898550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154102630 0x7fd154102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 --2- 172.21.15.3:0/2606898550 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd154106fe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 -- 172.21.15.3:0/2606898550 >> 172.21.15.3:0/2606898550 conn(0x7fd154079350 msgr2=0x7fd154079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 -- 172.21.15.3:0/2606898550 shutdown_connections 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.544+0000 7fd159044700 1 -- 172.21.15.3:0/2606898550 wait complete. 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.545+0000 7fd159044700 1 Processor -- start 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 -- start start 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154102630 0x7fd15419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd15419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1541076c0 0x7fd1541a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:10.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd154112580 con 0x7fd154102fe0 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd154112400 con 0x7fd154102630 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd159044700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd154112700 con 0x7fd1541076c0 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd15419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd15419afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34114/0 (socket says 172.21.15.3:34114) 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 -- 172.21.15.3:0/1150404792 learned_addr learned my addr 172.21.15.3:0/1150404792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd152d9d700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154102630 0x7fd15419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 -- 172.21.15.3:0/1150404792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1541076c0 msgr2=0x7fd1541a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15359e700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1541076c0 0x7fd1541a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:10.177 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1541076c0 0x7fd1541a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 -- 172.21.15.3:0/1150404792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154102630 msgr2=0x7fd15419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154102630 0x7fd15419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 -- 172.21.15.3:0/1150404792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1541a6020 con 0x7fd154102fe0 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd152d9d700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd154102630 0x7fd15419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.546+0000 7fd15259c700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd15419afa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006010 tx=0x7fd13c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.547+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c027070 con 0x7fd154102fe0 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.547+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1541a61f0 con 0x7fd154102fe0 2024-05-08T05:52:10.178 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.547+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd13c004120 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.547+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13c00f460 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.547+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1541a66a0 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.547+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd154063bb0 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.549+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fd13c02a070 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd14bfff700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1400b2990 0x7fd1400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd152d9d700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1400b2990 0x7fd1400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fd13c0e3e80 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd14bfff700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] conn(0x7fd1400bdbf0 0x7fd1400c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 --> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd13c00fe50 con 0x7fd1400bdbf0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fd13c0e42d0 con 0x7fd154102fe0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.550+0000 7fd15359e700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] conn(0x7fd1400bdbf0 0x7fd1400c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:10.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.551+0000 7fd152d9d700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1400b2990 0x7fd1400b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd144005950 tx=0x7fd144005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.551+0000 7fd15359e700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] conn(0x7fd1400bdbf0 0x7fd1400c0090 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.551+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== osd.0 v2:172.21.15.3:6802/3730465717 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd13c00fe50 con 0x7fd1400bdbf0 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.568+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 --> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd154064030 con 0x7fd1400bdbf0 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.568+0000 7fd14bfff700 1 -- 172.21.15.3:0/1150404792 <== osd.0 v2:172.21.15.3:6802/3730465717 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd154064030 con 0x7fd1400bdbf0 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] conn(0x7fd1400bdbf0 msgr2=0x7fd1400c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] conn(0x7fd1400bdbf0 0x7fd1400c0090 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1400b2990 msgr2=0x7fd1400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fd1400b2990 0x7fd1400b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd144005950 tx=0x7fd144005f50 comp rx=0 tx=0).stop 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 msgr2=0x7fd15419afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 --2- 172.21.15.3:0/1150404792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd154102fe0 0x7fd15419afa0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006010 tx=0x7fd13c003730 comp rx=0 tx=0).stop 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd15359e700 1 -- 172.21.15.3:0/1150404792 reap_dead start 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 shutdown_connections 2024-05-08T05:52:10.180 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 >> 172.21.15.3:0/1150404792 conn(0x7fd154079350 msgr2=0x7fd154100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:10.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 shutdown_connections 2024-05-08T05:52:10.181 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:09.569+0000 7fd159044700 1 -- 172.21.15.3:0/1150404792 wait complete. 2024-05-08T05:52:10.781 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:10 smithi003 ceph-mon[30137]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:10.811 INFO:teuthology.orchestra.run.smithi003.stdout:133143986254 2024-05-08T05:52:10.812 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T05:52:11.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:10 smithi089 ceph-mon[34495]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:11.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:10 smithi104 ceph-mon[34559]: pgmap v367: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:11.816 INFO:teuthology.orchestra.run.smithi003.stdout:77309411414 2024-05-08T05:52:11.817 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T05:52:12.107 INFO:teuthology.orchestra.run.smithi003.stdout:201863462973 2024-05-08T05:52:12.107 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T05:52:12.615 INFO:teuthology.orchestra.run.smithi003.stdout:34359738467 2024-05-08T05:52:12.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T05:52:12.639 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:12 smithi003 ceph-mon[30137]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:12 smithi089 ceph-mon[34495]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:12 smithi104 ceph-mon[34559]: pgmap v368: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:13.044 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:13.081 INFO:teuthology.orchestra.run.smithi003.stdout:227633266746 2024-05-08T05:52:13.081 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T05:52:13.315 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:13.316 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:13.317 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 -- 172.21.15.3:0/1241146244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 msgr2=0x7f25a810f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 --2- 172.21.15.3:0/1241146244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a810f9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2598009990 tx=0x7f2598009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 -- 172.21.15.3:0/1241146244 shutdown_connections 2024-05-08T05:52:13.567 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 --2- 172.21.15.3:0/1241146244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a810fef0 0x7f25a81122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 --2- 172.21.15.3:0/1241146244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a806bc50 0x7f25a806c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 --2- 172.21.15.3:0/1241146244 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a810f9b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 -- 172.21.15.3:0/1241146244 >> 172.21.15.3:0/1241146244 conn(0x7f25a8079440 msgr2=0x7f25a8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 -- 172.21.15.3:0/1241146244 shutdown_connections 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.565+0000 7f25afa73700 1 -- 172.21.15.3:0/1241146244 wait complete. 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 Processor -- start 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 -- start start 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a806bc50 0x7f25a819bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a819c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a810fef0 0x7f25a81a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f25a81158b0 con 0x7f25a806c570 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f25a8115730 con 0x7f25a806bc50 2024-05-08T05:52:13.568 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25afa73700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f25a8115a30 con 0x7f25a810fef0 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ad00e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a819c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ad00e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a819c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37948/0 (socket says 172.21.15.3:37948) 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ad00e700 1 -- 172.21.15.3:0/595103770 learned_addr learned my addr 172.21.15.3:0/595103770 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ae010700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a810fef0 0x7f25a81a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ad80f700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a806bc50 0x7f25a819bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ad00e700 1 -- 172.21.15.3:0/595103770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a810fef0 msgr2=0x7f25a81a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.566+0000 7f25ad00e700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a810fef0 0x7f25a81a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25ad00e700 1 -- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a806bc50 msgr2=0x7f25a819bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25ad00e700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a806bc50 0x7f25a819bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25ad00e700 1 -- 172.21.15.3:0/595103770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a81a0d80 con 0x7f25a806c570 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25ad80f700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f25a806bc50 0x7f25a819bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25ae010700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25a810fef0 0x7f25a81a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25ad00e700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a819c300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f25980057c0 tx=0x7f25980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598027070 con 0x7f25a806c570 2024-05-08T05:52:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2598004020 con 0x7f25a806c570 2024-05-08T05:52:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a81a1010 con 0x7f25a806c570 2024-05-08T05:52:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598022bf0 con 0x7f25a806c570 2024-05-08T05:52:13.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.567+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a81a88f0 con 0x7f25a806c570 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.568+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25a8063c30 con 0x7f25a806c570 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.570+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f259802a070 con 0x7f25a806c570 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.570+0000 7f259effd700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25940b2910 0x7f25940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f25ad80f700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25940b2910 0x7f25940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f25980e4190 con 0x7f25a806c570 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f259effd700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] conn(0x7f25940bdb70 0x7f25940c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 --> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25940c06e0 con 0x7f25940bdb70 2024-05-08T05:52:13.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f25980e45e0 con 0x7f25a806c570 2024-05-08T05:52:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f25ad80f700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25940b2910 0x7f25940b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005fd0 tx=0x7f25a4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.571+0000 7f25ae010700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] conn(0x7f25940bdb70 0x7f25940c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.572+0000 7f25ae010700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] conn(0x7f25940bdb70 0x7f25940c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:13.572 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.572+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== osd.10 v2:172.21.15.89:6824/3735647304 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25940c06e0 con 0x7f25940bdb70 2024-05-08T05:52:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.592+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 --> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f25a80640b0 con 0x7f25940bdb70 2024-05-08T05:52:13.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f259effd700 1 -- 172.21.15.3:0/595103770 <== osd.10 v2:172.21.15.89:6824/3735647304 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f25a80640b0 con 0x7f25940bdb70 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] conn(0x7f25940bdb70 msgr2=0x7f25940c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] conn(0x7f25940bdb70 0x7f25940c0010 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25940b2910 msgr2=0x7f25940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f25940b2910 0x7f25940b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005fd0 tx=0x7f25a4005e20 comp rx=0 tx=0).stop 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 msgr2=0x7f25a819c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 --2- 172.21.15.3:0/595103770 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f25a806c570 0x7f25a819c300 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f25980057c0 tx=0x7f25980057f0 comp rx=0 tx=0).stop 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25ae010700 1 -- 172.21.15.3:0/595103770 reap_dead start 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 shutdown_connections 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 >> 172.21.15.3:0/595103770 conn(0x7f25a8079440 msgr2=0x7f25a8111360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 shutdown_connections 2024-05-08T05:52:13.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:13.593+0000 7f25afa73700 1 -- 172.21.15.3:0/595103770 wait complete. 2024-05-08T05:52:14.908 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:14 smithi003 ceph-mon[30137]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:14 smithi089 ceph-mon[34495]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:15.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:14 smithi104 ceph-mon[34559]: pgmap v369: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:16.769 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:16 smithi003 ceph-mon[30137]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:16 smithi089 ceph-mon[34495]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:16 smithi104 ceph-mon[34559]: pgmap v370: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:18.371 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:18.783 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:18 smithi003 ceph-mon[30137]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:19.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:18 smithi089 ceph-mon[34495]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:19.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:18 smithi104 ceph-mon[34559]: pgmap v371: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:19.355 INFO:teuthology.orchestra.run.smithi003.stdout:274877906992 2024-05-08T05:52:19.355 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T05:52:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 -- 172.21.15.3:0/1917083014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806c3d0 msgr2=0x7ff1f8110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 --2- 172.21.15.3:0/1917083014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806c3d0 0x7ff1f8110320 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b0a0 tx=0x7ff1f000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 -- 172.21.15.3:0/1917083014 shutdown_connections 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 --2- 172.21.15.3:0/1917083014 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806c3d0 0x7ff1f8110320 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 --2- 172.21.15.3:0/1917083014 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f8109500 0x7ff1f81098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 --2- 172.21.15.3:0/1917083014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f806ba10 0x7ff1f806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 -- 172.21.15.3:0/1917083014 >> 172.21.15.3:0/1917083014 conn(0x7ff1f8079310 msgr2=0x7ff1f8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 -- 172.21.15.3:0/1917083014 shutdown_connections 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 -- 172.21.15.3:0/1917083014 wait complete. 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 Processor -- start 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.008+0000 7ff1fe906700 1 -- start start 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fe906700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 0x7ff1f819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:19.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fe906700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f806c3d0 0x7ff1f819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fe906700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f8109500 0x7ff1f8199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fe906700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113610 con 0x7ff1f806ba10 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fe906700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113490 con 0x7ff1f806c3d0 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fe906700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f8113790 con 0x7ff1f8109500 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 0x7ff1f819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 0x7ff1f819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37970/0 (socket says 172.21.15.3:37970) 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 -- 172.21.15.3:0/3677544442 learned_addr learned my addr 172.21.15.3:0/3677544442 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1fcea3700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f8109500 0x7ff1f8199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:19.422 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f77fe700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f806c3d0 0x7ff1f819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stdout:107374182485 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 -- 172.21.15.3:0/3677544442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f8109500 msgr2=0x7ff1f8199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f8109500 0x7ff1f8199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 -- 172.21.15.3:0/3677544442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f806c3d0 msgr2=0x7ff1f819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f806c3d0 0x7ff1f819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 -- 172.21.15.3:0/3677544442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f8199fd0 con 0x7ff1f806ba10 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f7fff700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 0x7ff1f819f6b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c920 tx=0x7ff1e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.009+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8007b00 con 0x7ff1f806ba10 2024-05-08T05:52:19.423 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.010+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e8004ce0 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.010+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f819a2c0 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.010+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e8005550 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.010+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f81a6580 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.010+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f806a190 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.012+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7ff1e8007ca0 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.013+0000 7ff1f57fa700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff1e00b2910 0x7ff1e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.013+0000 7ff1f77fe700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff1e00b2910 0x7ff1e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.013+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ff1e80e2600 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.013+0000 7ff1f77fe700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff1e00b2910 0x7ff1e00b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005ea0 tx=0x7ff1ec005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.016+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff1e8014030 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.232+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff1f819ada0 con 0x7ff1f806ba10 2024-05-08T05:52:19.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.232+0000 7ff1f57fa700 1 -- 172.21.15.3:0/3677544442 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1e80a9e30 con 0x7ff1f806ba10 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff1e00b2910 msgr2=0x7ff1e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff1e00b2910 0x7ff1e00b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec005ea0 tx=0x7ff1ec005e30 comp rx=0 tx=0).stop 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 msgr2=0x7ff1f819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 0x7ff1f819f6b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff1e800c920 tx=0x7ff1e800cc30 comp rx=0 tx=0).stop 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 shutdown_connections 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1f8109500 0x7ff1f8199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ff1e00b2910 0x7ff1e00b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1f806ba10 0x7ff1f819f6b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 --2- 172.21.15.3:0/3677544442 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1f806c3d0 0x7ff1f819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 >> 172.21.15.3:0/3677544442 conn(0x7ff1f8079310 msgr2=0x7ff1f8100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 shutdown_connections 2024-05-08T05:52:19.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:19.234+0000 7ff1fe906700 1 -- 172.21.15.3:0/3677544442 wait complete. 2024-05-08T05:52:19.872 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:19 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3677544442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:52:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:19 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3677544442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:52:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:19 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3677544442' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:52:20.178 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:20.193 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:20.894 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:20 smithi003 ceph-mon[30137]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:21.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:20 smithi089 ceph-mon[34495]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:21.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:20 smithi104 ceph-mon[34559]: pgmap v372: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:21.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 -- 172.21.15.3:0/2025701186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8228108880 msgr2=0x7f8228110400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 --2- 172.21.15.3:0/2025701186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8228108880 0x7f8228110400 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f822000b0d0 tx=0x7f822000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 -- 172.21.15.3:0/2025701186 shutdown_connections 2024-05-08T05:52:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 --2- 172.21.15.3:0/2025701186 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8228108880 0x7f8228110400 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 --2- 172.21.15.3:0/2025701186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f82281037b0 0x7f8228103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 --2- 172.21.15.3:0/2025701186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 0x7f8228108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 -- 172.21.15.3:0/2025701186 >> 172.21.15.3:0/2025701186 conn(0x7f822807a530 msgr2=0x7f822807a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 -- 172.21.15.3:0/2025701186 shutdown_connections 2024-05-08T05:52:21.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.636+0000 7f822e309700 1 -- 172.21.15.3:0/2025701186 wait complete. 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 Processor -- start 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 -- start start 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 0x7f822819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 0x7f822819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8228108880 0x7f8228199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f82281136a0 con 0x7f82281037b0 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8228113520 con 0x7f8228108880 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822e309700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8228113820 con 0x7f8228104160 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 0x7f822819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.640 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 0x7f822819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37988/0 (socket says 172.21.15.3:37988) 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 -- 172.21.15.3:0/1637107767 learned_addr learned my addr 172.21.15.3:0/1637107767 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f822c8a6700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8228108880 0x7f8228199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f821ffff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 0x7f822819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 -- 172.21.15.3:0/1637107767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 msgr2=0x7f822819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 0x7f822819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 -- 172.21.15.3:0/1637107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8228108880 msgr2=0x7f8228199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8228108880 0x7f8228199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f8227fff700 1 -- 172.21.15.3:0/1637107767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822819a060 con 0x7f82281037b0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.637+0000 7f821ffff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 0x7f822819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.638+0000 7f8227fff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 0x7f822819f6b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f821800d950 tx=0x7f821800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.638+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218016070 con 0x7f82281037b0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.638+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822819a350 con 0x7f82281037b0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.638+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8218009df0 con 0x7f82281037b0 2024-05-08T05:52:21.641 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.638+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821800f3c0 con 0x7f82281037b0 2024-05-08T05:52:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.638+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281a6510 con 0x7f82281037b0 2024-05-08T05:52:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.639+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8228104c10 con 0x7f82281037b0 2024-05-08T05:52:21.642 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.641+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f8218019070 con 0x7f82281037b0 2024-05-08T05:52:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.642+0000 7f8225ffb700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f82140b2910 0x7f82140b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.642+0000 7f821ffff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f82140b2910 0x7f82140b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.642+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f82180e2380 con 0x7f82281037b0 2024-05-08T05:52:21.643 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.643+0000 7f821ffff700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f82140b2910 0x7f82140b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8210005fd0 tx=0x7f8210005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:21.646 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.646+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82180a9b50 con 0x7f82281037b0 2024-05-08T05:52:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.869+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f822802e510 con 0x7f82281037b0 2024-05-08T05:52:21.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.870+0000 7f8225ffb700 1 -- 172.21.15.3:0/1637107767 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f82180a9450 con 0x7f82281037b0 2024-05-08T05:52:21.870 INFO:teuthology.orchestra.run.smithi003.stdout:55834574942 2024-05-08T05:52:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f82140b2910 msgr2=0x7f82140b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f82140b2910 0x7f82140b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8210005fd0 tx=0x7f8210005e20 comp rx=0 tx=0).stop 2024-05-08T05:52:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 msgr2=0x7f822819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 0x7f822819f6b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f821800d950 tx=0x7f821800dc60 comp rx=0 tx=0).stop 2024-05-08T05:52:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 shutdown_connections 2024-05-08T05:52:21.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8228108880 0x7f8228199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f82140b2910 0x7f82140b4dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f82281037b0 0x7f822819f6b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 --2- 172.21.15.3:0/1637107767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8228104160 0x7f822819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 >> 172.21.15.3:0/1637107767 conn(0x7f822807a530 msgr2=0x7f8228101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 shutdown_connections 2024-05-08T05:52:21.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:21.872+0000 7f822e309700 1 -- 172.21.15.3:0/1637107767 wait complete. 2024-05-08T05:52:22.040 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:22.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:22 smithi003 ceph-mon[30137]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:22.711 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:22 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1637107767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:52:22.741 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574940 got 55834574942 for osd.1 2024-05-08T05:52:22.742 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:23.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:22 smithi089 ceph-mon[34495]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:23.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:22 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1637107767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:52:23.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:22 smithi104 ceph-mon[34559]: pgmap v373: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:23.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:22 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1637107767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:52:23.154 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:23.155 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:23.165 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182481 got 107374182485 for osd.3 2024-05-08T05:52:23.165 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:24.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 -- 172.21.15.3:0/3462437155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4108870 msgr2=0x7f16f41103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:24.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 --2- 172.21.15.3:0/3462437155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4108870 0x7f16f41103f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00b0d0 tx=0x7f16ec00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 -- 172.21.15.3:0/3462437155 shutdown_connections 2024-05-08T05:52:24.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 --2- 172.21.15.3:0/3462437155 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4108870 0x7f16f41103f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 --2- 172.21.15.3:0/3462437155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f4103780 0x7f16f4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 --2- 172.21.15.3:0/3462437155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4104130 0x7f16f4108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 -- 172.21.15.3:0/3462437155 >> 172.21.15.3:0/3462437155 conn(0x7f16f407a610 msgr2=0x7f16f407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:24.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 -- 172.21.15.3:0/3462437155 shutdown_connections 2024-05-08T05:52:24.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 -- 172.21.15.3:0/3462437155 wait complete. 2024-05-08T05:52:24.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.029+0000 7f16fa149700 1 Processor -- start 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 -- start start 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4103780 0x7f16f419f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 0x7f16f419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f4108870 0x7f16f41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4113640 con 0x7f16f4104130 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41134c0 con 0x7f16f4103780 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16fa149700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f16f41137c0 con 0x7f16f4108870 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 0x7f16f419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 0x7f16f419fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38668/0 (socket says 172.21.15.3:38668) 2024-05-08T05:52:24.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 -- 172.21.15.3:0/3757558164 learned_addr learned my addr 172.21.15.3:0/3757558164 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f37fe700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4103780 0x7f16f419f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f3fff700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f4108870 0x7f16f41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 -- 172.21.15.3:0/3757558164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f4108870 msgr2=0x7f16f41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f4108870 0x7f16f41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 -- 172.21.15.3:0/3757558164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4103780 msgr2=0x7f16f419f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4103780 0x7f16f419f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 -- 172.21.15.3:0/3757558164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f419a080 con 0x7f16f4104130 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f37fe700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4103780 0x7f16f419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.030+0000 7f16f2ffd700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 0x7f16f419fc10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16e800d950 tx=0x7f16e800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.031+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8016070 con 0x7f16f4104130 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.031+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f419a370 con 0x7f16f4104130 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.031+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16e8010460 con 0x7f16f4104130 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.031+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e800f620 con 0x7f16f4104130 2024-05-08T05:52:24.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.031+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f41a6530 con 0x7f16f4104130 2024-05-08T05:52:24.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.032+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16f4104c00 con 0x7f16f4104130 2024-05-08T05:52:24.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.033+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f16e8019070 con 0x7f16f4104130 2024-05-08T05:52:24.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.034+0000 7f16f0ff9700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f16dc0b2910 0x7f16dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:24.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.034+0000 7f16f37fe700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f16dc0b2910 0x7f16dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:24.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.034+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f16e80e23a0 con 0x7f16f4104130 2024-05-08T05:52:24.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.034+0000 7f16f37fe700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f16dc0b2910 0x7f16dc0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f16e4005950 tx=0x7f16e4005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:24.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.037+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16e80a9bd0 con 0x7f16f4104130 2024-05-08T05:52:24.258 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.258+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f16f402e510 con 0x7f16f4104130 2024-05-08T05:52:24.259 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.258+0000 7f16f0ff9700 1 -- 172.21.15.3:0/3757558164 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f16e80a94d0 con 0x7f16f4104130 2024-05-08T05:52:24.259 INFO:teuthology.orchestra.run.smithi003.stdout:300647710764 2024-05-08T05:52:24.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.260+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f16dc0b2910 msgr2=0x7f16dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:24.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.260+0000 7f16fa149700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f16dc0b2910 0x7f16dc0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f16e4005950 tx=0x7f16e4005f50 comp rx=0 tx=0).stop 2024-05-08T05:52:24.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 msgr2=0x7f16f419fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:24.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 0x7f16f419fc10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f16e800d950 tx=0x7f16e800dd10 comp rx=0 tx=0).stop 2024-05-08T05:52:24.261 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 shutdown_connections 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16f4108870 0x7f16f41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f16f4103780 0x7f16f419f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f16dc0b2910 0x7f16dc0b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 --2- 172.21.15.3:0/3757558164 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f16f4104130 0x7f16f419fc10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 >> 172.21.15.3:0/3757558164 conn(0x7f16f407a610 msgr2=0x7f16f41014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 shutdown_connections 2024-05-08T05:52:24.262 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.261+0000 7f16fa149700 1 -- 172.21.15.3:0/3757558164 wait complete. 2024-05-08T05:52:24.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:24 smithi003 ceph-mon[30137]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:24.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:24 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3757558164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:52:24.998 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710761 got 300647710764 for osd.11 2024-05-08T05:52:24.998 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:25.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:24 smithi089 ceph-mon[34495]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:25.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:24 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3757558164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:52:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:24 smithi104 ceph-mon[34559]: pgmap v374: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:24 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3757558164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:52:25.849 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:25 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/421524634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:52:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:25 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/421524634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:52:26.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:25 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/421524634' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 -- 172.21.15.3:0/3561669276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8103750 msgr2=0x7f6ca8103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/3561669276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8103750 0x7f6ca8103b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6c980099c0 tx=0x7f6c98009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 -- 172.21.15.3:0/3561669276 shutdown_connections 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/3561669276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ca8108850 0x7f6ca81103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/3561669276 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8103750 0x7f6ca8103b30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/3561669276 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 0x7f6ca8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 -- 172.21.15.3:0/3561669276 >> 172.21.15.3:0/3561669276 conn(0x7f6ca807a600 msgr2=0x7f6ca807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 -- 172.21.15.3:0/3561669276 shutdown_connections 2024-05-08T05:52:26.069 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.053+0000 7f6cad2a2700 1 -- 172.21.15.3:0/3561669276 wait complete. 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.054+0000 7f6cad2a2700 1 Processor -- start 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.054+0000 7f6cad2a2700 1 -- start start 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.054+0000 7f6cad2a2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ca8103750 0x7f6ca819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.054+0000 7f6cad2a2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 0x7f6ca819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.054+0000 7f6cad2a2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 0x7f6ca8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6cad2a2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca8113620 con 0x7f6ca8108850 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6cad2a2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81134a0 con 0x7f6ca8104100 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6cad2a2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca81137a0 con 0x7f6ca8103750 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 0x7f6ca8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 0x7f6ca8199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38694/0 (socket says 172.21.15.3:38694) 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 -- 172.21.15.3:0/1189099146 learned_addr learned my addr 172.21.15.3:0/1189099146 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca6ffd700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ca8103750 0x7f6ca819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca67fc700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 0x7f6ca819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:26.070 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 -- 172.21.15.3:0/1189099146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ca8103750 msgr2=0x7f6ca819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ca8103750 0x7f6ca819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 -- 172.21.15.3:0/1189099146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 msgr2=0x7f6ca819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 0x7f6ca819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 -- 172.21.15.3:0/1189099146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca819a040 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca67fc700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 0x7f6ca819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6ca77fe700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 0x7f6ca8199860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0007d70 tx=0x7f6ca000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0016020 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca000ad40 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca819a330 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.055+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00115e0 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.056+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca81a64f0 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.056+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca810ba90 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.058+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f6ca0019020 con 0x7f6ca8108850 2024-05-08T05:52:26.071 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.059+0000 7f6c9ffff700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6c940b2910 0x7f6c940b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:26.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.059+0000 7f6ca6ffd700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6c940b2910 0x7f6c940b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:26.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.059+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6ca00e29a0 con 0x7f6ca8108850 2024-05-08T05:52:26.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.059+0000 7f6ca6ffd700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6c940b2910 0x7f6c940b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:26.072 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.062+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6ca0020020 con 0x7f6ca8108850 2024-05-08T05:52:26.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.279+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6ca810b470 con 0x7f6ca8108850 2024-05-08T05:52:26.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.280+0000 7f6c9ffff700 1 -- 172.21.15.3:0/1189099146 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6ca00aa2e0 con 0x7f6ca8108850 2024-05-08T05:52:26.280 INFO:teuthology.orchestra.run.smithi003.stdout:180388626504 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6c940b2910 msgr2=0x7f6c940b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6c940b2910 0x7f6c940b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6c98009990 tx=0x7f6c98009650 comp rx=0 tx=0).stop 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 msgr2=0x7f6ca8199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 0x7f6ca8199860 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0007d70 tx=0x7f6ca000a340 comp rx=0 tx=0).stop 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 shutdown_connections 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6c940b2910 0x7f6c940b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6ca8108850 0x7f6ca8199860 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ca8103750 0x7f6ca819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 --2- 172.21.15.3:0/1189099146 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6ca8104100 0x7f6ca819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.282+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 >> 172.21.15.3:0/1189099146 conn(0x7f6ca807a600 msgr2=0x7f6ca8101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.283+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 shutdown_connections 2024-05-08T05:52:26.283 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:26.283+0000 7f6cad2a2700 1 -- 172.21.15.3:0/1189099146 wait complete. 2024-05-08T05:52:26.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:26 smithi003 ceph-mon[30137]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:26.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:26 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1189099146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:52:27.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:26 smithi089 ceph-mon[34495]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:27.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:26 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1189099146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:52:27.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:26 smithi104 ceph-mon[34559]: pgmap v375: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:26 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1189099146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:52:27.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 -- 172.21.15.3:0/1496005211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c103750 msgr2=0x7f412c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 --2- 172.21.15.3:0/1496005211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c103750 0x7f412c103b30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f41140099c0 tx=0x7f4114009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 -- 172.21.15.3:0/1496005211 shutdown_connections 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 --2- 172.21.15.3:0/1496005211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f412c108850 0x7f412c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 --2- 172.21.15.3:0/1496005211 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c103750 0x7f412c103b30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 --2- 172.21.15.3:0/1496005211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c104100 0x7f412c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 -- 172.21.15.3:0/1496005211 >> 172.21.15.3:0/1496005211 conn(0x7f412c07a600 msgr2=0x7f412c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 -- 172.21.15.3:0/1496005211 shutdown_connections 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.387+0000 7f41330a0700 1 -- 172.21.15.3:0/1496005211 wait complete. 2024-05-08T05:52:27.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 Processor -- start 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 -- start start 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c103750 0x7f412c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 0x7f412c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f412c108850 0x7f412c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f412c113670 con 0x7f412c104100 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f412c1134f0 con 0x7f412c103750 2024-05-08T05:52:27.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f41330a0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f412c1137f0 con 0x7f412c108850 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f412bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 0x7f412c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f412bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 0x7f412c19fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38674/0 (socket says 172.21.15.3:38674) 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f412bfff700 1 -- 172.21.15.3:0/421524634 learned_addr learned my addr 172.21.15.3:0/421524634 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f413163d700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f412c108850 0x7f412c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.388+0000 7f4130e3c700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c103750 0x7f412c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f412bfff700 1 -- 172.21.15.3:0/421524634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f412c108850 msgr2=0x7f412c1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f412bfff700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f412c108850 0x7f412c1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f412bfff700 1 -- 172.21.15.3:0/421524634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c103750 msgr2=0x7f412c19f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f412bfff700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c103750 0x7f412c19f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f412bfff700 1 -- 172.21.15.3:0/421524634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c19a050 con 0x7f412c104100 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f4130e3c700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c103750 0x7f412c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f412bfff700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 0x7f412c19fc70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f411c0060b0 tx=0x7f411c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:27.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c01a070 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c19a340 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c00ddd0 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c014720 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.389+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c1a6600 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.390+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c10ba90 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.392+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f411c01d070 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.392+0000 7f4129ffb700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f41180b2910 0x7f41180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.392+0000 7f4130e3c700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f41180b2910 0x7f41180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.393+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f411c0e1ff0 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.393+0000 7f4130e3c700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f41180b2910 0x7f41180b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4114000c00 tx=0x7f4114005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.397+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f411c010070 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.616+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f412c10b470 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.616+0000 7f4129ffb700 1 -- 172.21.15.3:0/421524634 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f411c0a9820 con 0x7f412c104100 2024-05-08T05:52:27.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f41180b2910 msgr2=0x7f41180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f41180b2910 0x7f41180b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4114000c00 tx=0x7f4114005fb0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 msgr2=0x7f412c19fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 0x7f412c19fc70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f411c0060b0 tx=0x7f411c00d3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 shutdown_connections 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f412c108850 0x7f412c1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f412c103750 0x7f412c19f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f41180b2910 0x7f41180b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 --2- 172.21.15.3:0/421524634 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f412c104100 0x7f412c19fc70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 >> 172.21.15.3:0/421524634 conn(0x7f412c07a600 msgr2=0x7f412c101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 shutdown_connections 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:24.619+0000 7f41330a0700 1 -- 172.21.15.3:0/421524634 wait complete. 2024-05-08T05:52:27.745 INFO:teuthology.orchestra.run.smithi003.stdout:154618822732 2024-05-08T05:52:28.343 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822726 got 154618822732 for osd.5 2024-05-08T05:52:28.343 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:28.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:28 smithi003 ceph-mon[30137]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:29.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:28 smithi089 ceph-mon[34495]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:29.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:28 smithi104 ceph-mon[34559]: pgmap v376: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:29.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 -- 172.21.15.3:0/2263093711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806bb40 msgr2=0x7f2ed8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 --2- 172.21.15.3:0/2263093711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806bb40 0x7f2ed8110330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009990 tx=0x7f2ed0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 -- 172.21.15.3:0/2263093711 shutdown_connections 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 --2- 172.21.15.3:0/2263093711 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806bb40 0x7f2ed8110330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 --2- 172.21.15.3:0/2263093711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 0x7f2ed81098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 --2- 172.21.15.3:0/2263093711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ed806b180 0x7f2ed806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 -- 172.21.15.3:0/2263093711 >> 172.21.15.3:0/2263093711 conn(0x7f2ed8079350 msgr2=0x7f2ed8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 -- 172.21.15.3:0/2263093711 shutdown_connections 2024-05-08T05:52:29.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.202+0000 7f2edd585700 1 -- 172.21.15.3:0/2263093711 wait complete. 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 Processor -- start 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 -- start start 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 0x7f2ed819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ed806bb40 0x7f2ed819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 0x7f2ed8199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed8113640 con 0x7f2ed806b180 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81134c0 con 0x7f2ed8109510 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2edd585700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed81137c0 con 0x7f2ed806bb40 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2ed6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 0x7f2ed819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.203+0000 7f2ed6ffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 0x7f2ed819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38716/0 (socket says 172.21.15.3:38716) 2024-05-08T05:52:29.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 -- 172.21.15.3:0/791285792 learned_addr learned my addr 172.21.15.3:0/791285792 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed67fc700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ed806bb40 0x7f2ed819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed77fe700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 0x7f2ed8199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 -- 172.21.15.3:0/791285792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ed806bb40 msgr2=0x7f2ed819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ed806bb40 0x7f2ed819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 -- 172.21.15.3:0/791285792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 msgr2=0x7f2ed8199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 0x7f2ed8199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 -- 172.21.15.3:0/791285792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed819a080 con 0x7f2ed806b180 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed77fe700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 0x7f2ed8199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ed6ffd700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 0x7f2ed819f760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8007f10 tx=0x7f2ec800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800ddd0 con 0x7f2ed806b180 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed819a370 con 0x7f2ed806b180 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ec8004bb0 con 0x7f2ed806b180 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80046a0 con 0x7f2ed806b180 2024-05-08T05:52:29.206 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.204+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed81a6630 con 0x7f2ed806b180 2024-05-08T05:52:29.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.205+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed806a190 con 0x7f2ed806b180 2024-05-08T05:52:29.207 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.208+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7f2ec801c020 con 0x7f2ed806b180 2024-05-08T05:52:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.208+0000 7f2ecffff700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2ec40b2910 0x7f2ec40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.208+0000 7f2ed67fc700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2ec40b2910 0x7f2ec40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:29.208 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.208+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f2ec80e2a00 con 0x7f2ed806b180 2024-05-08T05:52:29.209 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.209+0000 7f2ed67fc700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2ec40b2910 0x7f2ec40b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80799c0 tx=0x7f2ec000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:29.211 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.212+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2ec8023030 con 0x7f2ed806b180 2024-05-08T05:52:29.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.428+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2ed802e510 con 0x7f2ed806b180 2024-05-08T05:52:29.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.429+0000 7f2ecffff700 1 -- 172.21.15.3:0/791285792 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ec8010070 con 0x7f2ed806b180 2024-05-08T05:52:29.429 INFO:teuthology.orchestra.run.smithi003.stdout:201863462978 2024-05-08T05:52:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2ec40b2910 msgr2=0x7f2ec40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2ec40b2910 0x7f2ec40b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80799c0 tx=0x7f2ec000b3c0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 msgr2=0x7f2ed819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 0x7f2ed819f760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8007f10 tx=0x7f2ec800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 shutdown_connections 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2ed8109510 0x7f2ed8199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f2ec40b2910 0x7f2ec40b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f2ed806b180 0x7f2ed819f760 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 --2- 172.21.15.3:0/791285792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ed806bb40 0x7f2ed819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 >> 172.21.15.3:0/791285792 conn(0x7f2ed8079350 msgr2=0x7f2ed8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 shutdown_connections 2024-05-08T05:52:29.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:29.431+0000 7f2edd585700 1 -- 172.21.15.3:0/791285792 wait complete. 2024-05-08T05:52:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:29 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/791285792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:52:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:29 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/791285792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:52:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:29 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/791285792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:52:30.468 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626498 got 180388626504 for osd.6 2024-05-08T05:52:30.469 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:30.665 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:30.665 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:30.789 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:30 smithi003 ceph-mon[30137]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:31.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:30 smithi089 ceph-mon[34495]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:31.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:30 smithi104 ceph-mon[34559]: pgmap v377: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:32 smithi003 ceph-mon[30137]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:33.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:32 smithi089 ceph-mon[34495]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:33.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:32 smithi104 ceph-mon[34559]: pgmap v378: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:33.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:33 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/534519881' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:52:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:33 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/534519881' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:52:34.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:33 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/534519881' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:52:34.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:34 smithi003 ceph-mon[30137]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:34.764 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:34 smithi003 ceph-mon[30137]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 -- 172.21.15.3:0/511931380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 msgr2=0x7f238c102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 --2- 172.21.15.3:0/511931380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102030 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f23880099c0 tx=0x7f2388009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 -- 172.21.15.3:0/511931380 shutdown_connections 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 --2- 172.21.15.3:0/511931380 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f238c108890 0x7f238c110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 --2- 172.21.15.3:0/511931380 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102030 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 --2- 172.21.15.3:0/511931380 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f238c102600 0x7f238c108160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.972+0000 7f2393971700 1 -- 172.21.15.3:0/511931380 >> 172.21.15.3:0/511931380 conn(0x7f238c0789a0 msgr2=0x7f238c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.973+0000 7f2393971700 1 -- 172.21.15.3:0/511931380 shutdown_connections 2024-05-08T05:52:34.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.973+0000 7f2393971700 1 -- 172.21.15.3:0/511931380 wait complete. 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.973+0000 7f2393971700 1 Processor -- start 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.973+0000 7f2393971700 1 -- start start 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.973+0000 7f2393971700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.973+0000 7f2393971700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f238c102600 0x7f238c103110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2393971700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f238c108890 0x7f238c103650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2393971700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f238c113680 con 0x7f238c101c50 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2393971700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f238c113500 con 0x7f238c108890 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2393971700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f238c113800 con 0x7f238c102600 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51140/0 (socket says 172.21.15.3:51140) 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 -- 172.21.15.3:0/3850472673 learned_addr learned my addr 172.21.15.3:0/3850472673 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2390f0c700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f238c102600 0x7f238c103110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:34.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2391f0e700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f238c108890 0x7f238c103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 -- 172.21.15.3:0/3850472673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f238c102600 msgr2=0x7f238c103110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f238c102600 0x7f238c103110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 -- 172.21.15.3:0/3850472673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f238c108890 msgr2=0x7f238c103650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f238c108890 0x7f238c103650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 -- 172.21.15.3:0/3850472673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c1a8060 con 0x7f238c101c50 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f239170d700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102bd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2388006010 tx=0x7f23880045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388026070 con 0x7f238c101c50 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f238800f460 con 0x7f238c101c50 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c1a82f0 con 0x7f238c101c50 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388009e60 con 0x7f238c101c50 2024-05-08T05:52:34.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.974+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1a8740 con 0x7f238c101c50 2024-05-08T05:52:34.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.975+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c10bad0 con 0x7f238c101c50 2024-05-08T05:52:34.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.977+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f2388029070 con 0x7f238c101c50 2024-05-08T05:52:34.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.978+0000 7f23827fc700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f23780b2880 0x7f23780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:34.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.978+0000 7f2390f0c700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f23780b2880 0x7f23780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:34.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.978+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f23880e3210 con 0x7f238c101c50 2024-05-08T05:52:34.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.978+0000 7f2390f0c700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f23780b2880 0x7f23780b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f238c106070 tx=0x7f237c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:34.982 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:34.982+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2388030020 con 0x7f238c101c50 2024-05-08T05:52:35.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:34 smithi089 ceph-mon[34495]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:35.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:34 smithi089 ceph-mon[34495]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:52:35.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:34 smithi104 ceph-mon[34559]: pgmap v379: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:35.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:34 smithi104 ceph-mon[34559]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:52:35.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 -- 172.21.15.3:0/3476101121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64006efe0 msgr2=0x7fb64006f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 --2- 172.21.15.3:0/3476101121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64006efe0 0x7fb64006f3c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb630009990 tx=0x7fb630009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 -- 172.21.15.3:0/3476101121 shutdown_connections 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 --2- 172.21.15.3:0/3476101121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb64011cb50 0x7fb64011ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 --2- 172.21.15.3:0/3476101121 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64006efe0 0x7fb64006f3c0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb630009990 tx=0x7fb630009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 --2- 172.21.15.3:0/3476101121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb64006f900 0x7fb64011c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.233+0000 7fb64716c700 1 -- 172.21.15.3:0/3476101121 >> 172.21.15.3:0/3476101121 conn(0x7fb64006e6c0 msgr2=0x7fb64010c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.234+0000 7fb64716c700 1 -- 172.21.15.3:0/3476101121 shutdown_connections 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.234+0000 7fb64716c700 1 -- 172.21.15.3:0/3476101121 wait complete. 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.234+0000 7fb64716c700 1 Processor -- start 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 -- start start 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb64006f900 0x7fb6401a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 0x7fb6401a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6401ad630 0x7fb6401adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stdout:133143986259 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb6401225a0 con 0x7fb64011cb50 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb640122420 con 0x7fb6401ad630 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.235+0000 7fb64716c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb640122720 con 0x7fb64006f900 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 0x7fb6401a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb645709700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6401ad630 0x7fb6401adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 0x7fb6401a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51118/0 (socket says 172.21.15.3:51118) 2024-05-08T05:52:35.034 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 -- 172.21.15.3:0/534519881 learned_addr learned my addr 172.21.15.3:0/534519881 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb644f08700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb64006f900 0x7fb6401a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 -- 172.21.15.3:0/534519881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb64006f900 msgr2=0x7fb6401a8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb64006f900 0x7fb6401a8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 -- 172.21.15.3:0/534519881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6401ad630 msgr2=0x7fb6401adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6401ad630 0x7fb6401adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 -- 172.21.15.3:0/534519881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6401ae150 con 0x7fb64011cb50 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.236+0000 7fb63ffff700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 0x7fb6401a92f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb63800ba50 tx=0x7fb63800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.238+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638019070 con 0x7fb64011cb50 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.238+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63800cde0 con 0x7fb64011cb50 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.238+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb638007370 con 0x7fb64011cb50 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.239+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6401ae440 con 0x7fb64011cb50 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.239+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64010e0b0 con 0x7fb64011cb50 2024-05-08T05:52:35.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.241+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c005330 con 0x7fb64011cb50 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.243+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 17) ==== 149702+0+0 (secure 0 0 0) 0x7fb63801c070 con 0x7fb64011cb50 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.244+0000 7fb63dffb700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb6280b2a60 0x7fb6280b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.244+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fb6380e18b0 con 0x7fb64011cb50 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.245+0000 7fb644f08700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb6280b2a60 0x7fb6280b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.245+0000 7fb644f08700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb6280b2a60 0x7fb6280b4f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb6300098e0 tx=0x7fb630005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.249+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb638051df0 con 0x7fb64011cb50 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.466+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb62c0058f0 con 0x7fb64011cb50 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.467+0000 7fb63dffb700 1 -- 172.21.15.3:0/534519881 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb6380a91f0 con 0x7fb64011cb50 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb6280b2a60 msgr2=0x7fb6280b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb6280b2a60 0x7fb6280b4f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb6300098e0 tx=0x7fb630005e60 comp rx=0 tx=0).stop 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 msgr2=0x7fb6401a92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 0x7fb6401a92f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb63800ba50 tx=0x7fb63800bd60 comp rx=0 tx=0).stop 2024-05-08T05:52:35.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 shutdown_connections 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb6401ad630 0x7fb6401adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb64006f900 0x7fb6401a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fb6280b2a60 0x7fb6280b4f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 --2- 172.21.15.3:0/534519881 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb64011cb50 0x7fb6401a92f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 >> 172.21.15.3:0/534519881 conn(0x7fb64006e6c0 msgr2=0x7fb64011b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 shutdown_connections 2024-05-08T05:52:35.037 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:33.469+0000 7fb64716c700 1 -- 172.21.15.3:0/534519881 wait complete. 2024-05-08T05:52:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.198+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f238c10b4b0 con 0x7f238c101c50 2024-05-08T05:52:35.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.199+0000 7f23827fc700 1 -- 172.21.15.3:0/3850472673 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2388013070 con 0x7f238c101c50 2024-05-08T05:52:35.199 INFO:teuthology.orchestra.run.smithi003.stdout:274877906995 2024-05-08T05:52:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f23780b2880 msgr2=0x7f23780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f23780b2880 0x7f23780b4d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f238c106070 tx=0x7f237c009260 comp rx=0 tx=0).stop 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 msgr2=0x7f238c102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102bd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2388006010 tx=0x7f23880045e0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 shutdown_connections 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f238c108890 0x7f238c103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f23780b2880 0x7f23780b4d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f238c101c50 0x7f238c102bd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 --2- 172.21.15.3:0/3850472673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f238c102600 0x7f238c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 >> 172.21.15.3:0/3850472673 conn(0x7f238c0789a0 msgr2=0x7f238c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 shutdown_connections 2024-05-08T05:52:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:35.201+0000 7f2393971700 1 -- 172.21.15.3:0/3850472673 wait complete. 2024-05-08T05:52:35.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:35 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3850472673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:52:35.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986254 got 133143986259 for osd.4 2024-05-08T05:52:35.857 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:35 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3850472673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:52:36.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:35 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3850472673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 -- 172.21.15.3:0/996027791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad81023a0 msgr2=0x7fdad8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 --2- 172.21.15.3:0/996027791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad81023a0 0x7fdad8102820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009990 tx=0x7fdac0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 -- 172.21.15.3:0/996027791 shutdown_connections 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 --2- 172.21.15.3:0/996027791 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 0x7fdad810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 --2- 172.21.15.3:0/996027791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8108410 0x7fdad81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 --2- 172.21.15.3:0/996027791 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad81023a0 0x7fdad8102820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 -- 172.21.15.3:0/996027791 >> 172.21.15.3:0/996027791 conn(0x7fdad8079350 msgr2=0x7fdad8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:36.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 -- 172.21.15.3:0/996027791 shutdown_connections 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.047+0000 7fdade5dc700 1 -- 172.21.15.3:0/996027791 wait complete. 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 Processor -- start 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 -- start start 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad81023a0 0x7fdad819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 0x7fdad819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 0x7fdad8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8112630 con 0x7fdad8108410 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81124b0 con 0x7fdad8102d60 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdade5dc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81127b0 con 0x7fdad81023a0 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdadcb79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 0x7fdad8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdadcb79700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 0x7fdad8198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51166/0 (socket says 172.21.15.3:51166) 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdadcb79700 1 -- 172.21.15.3:0/762974541 learned_addr learned my addr 172.21.15.3:0/762974541 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.048+0000 7fdad7fff700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad81023a0 0x7fdad819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:36.300 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdacffff700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 0x7fdad819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdadcb79700 1 -- 172.21.15.3:0/762974541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad81023a0 msgr2=0x7fdad819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdadcb79700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad81023a0 0x7fdad819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdadcb79700 1 -- 172.21.15.3:0/762974541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 msgr2=0x7fdad819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdadcb79700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 0x7fdad819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdadcb79700 1 -- 172.21.15.3:0/762974541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad81991f0 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdacffff700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 0x7fdad819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdadcb79700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 0x7fdad8198a90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdad000b970 tx=0x7fdad000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad0019070 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad81994e0 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdad0014440 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad000ce60 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.049+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad81a57a0 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.050+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad810a980 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.051+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fdad001c070 con 0x7fdad8108410 2024-05-08T05:52:36.301 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.052+0000 7fdad5ffb700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdac40b2880 0x7fdac40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.052+0000 7fdad7fff700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdac40b2880 0x7fdac40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.052+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fdad00e1f20 con 0x7fdad8108410 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.053+0000 7fdad7fff700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdac40b2880 0x7fdac40b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006fd0 tx=0x7fdac8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.056+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdad00a9850 con 0x7fdad8108410 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stdout:253403070522 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.275+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fdad8199fc0 con 0x7fdad8108410 2024-05-08T05:52:36.302 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.275+0000 7fdad5ffb700 1 -- 172.21.15.3:0/762974541 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdad00a9150 con 0x7fdad8108410 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.277+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdac40b2880 msgr2=0x7fdac40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.277+0000 7fdade5dc700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdac40b2880 0x7fdac40b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdac8006fd0 tx=0x7fdac8008040 comp rx=0 tx=0).stop 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.277+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 msgr2=0x7fdad8198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.277+0000 7fdade5dc700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 0x7fdad8198a90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdad000b970 tx=0x7fdad000bd30 comp rx=0 tx=0).stop 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 shutdown_connections 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fdac40b2880 0x7fdac40b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fdad8108410 0x7fdad8198a90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad81023a0 0x7fdad819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 --2- 172.21.15.3:0/762974541 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fdad8102d60 0x7fdad819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 >> 172.21.15.3:0/762974541 conn(0x7fdad8079350 msgr2=0x7fdad8101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 shutdown_connections 2024-05-08T05:52:36.303 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:36.278+0000 7fdade5dc700 1 -- 172.21.15.3:0/762974541 wait complete. 2024-05-08T05:52:36.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:36 smithi003 ceph-mon[30137]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:36.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:36 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/762974541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:52:37.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:36 smithi089 ceph-mon[34495]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:37.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:36 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/762974541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:52:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:36 smithi104 ceph-mon[34559]: pgmap v380: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:37.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:36 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/762974541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:52:37.051 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906992 got 274877906995 for osd.10 2024-05-08T05:52:37.051 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:37.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462973 got 201863462978 for osd.7 2024-05-08T05:52:37.475 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:38.301 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070516 got 253403070522 for osd.9 2024-05-08T05:52:38.301 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:38.832 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:38 smithi003 ceph-mon[30137]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-05-08T05:52:38.832 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:38.832 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:38 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:39.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:38 smithi089 ceph-mon[34495]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-05-08T05:52:39.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:39.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:38 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:38 smithi104 ceph-mon[34559]: pgmap v381: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-05-08T05:52:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:39.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:38 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:39.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 -- 172.21.15.3:0/1359509205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c102d50 msgr2=0x7f6f8c10f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 --2- 172.21.15.3:0/1359509205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c102d50 0x7f6f8c10f240 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b0a0 tx=0x7f6f8400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 -- 172.21.15.3:0/1359509205 shutdown_connections 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 --2- 172.21.15.3:0/1359509205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c102d50 0x7f6f8c10f240 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 --2- 172.21.15.3:0/1359509205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c108400 0x7f6f8c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 --2- 172.21.15.3:0/1359509205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102390 0x7f6f8c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 -- 172.21.15.3:0/1359509205 >> 172.21.15.3:0/1359509205 conn(0x7f6f8c079310 msgr2=0x7f6f8c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 -- 172.21.15.3:0/1359509205 shutdown_connections 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 -- 172.21.15.3:0/1359509205 wait complete. 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 Processor -- start 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.272+0000 7f6f9346b700 1 -- start start 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f9346b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c102390 0x7f6f8c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f9346b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102d50 0x7f6f8c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f9346b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c108400 0x7f6f8c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f9346b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c112520 con 0x7f6f8c108400 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f9346b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1123a0 con 0x7f6f8c102d50 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f9346b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c1126a0 con 0x7f6f8c102390 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91a08700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c108400 0x7f6f8c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91a08700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c108400 0x7f6f8c199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51214/0 (socket says 172.21.15.3:51214) 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f90a06700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102d50 0x7f6f8c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91a08700 1 -- 172.21.15.3:0/151806865 learned_addr learned my addr 172.21.15.3:0/151806865 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91207700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c102390 0x7f6f8c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91207700 1 -- 172.21.15.3:0/151806865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102d50 msgr2=0x7f6f8c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91207700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102d50 0x7f6f8c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.276 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91207700 1 -- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c108400 msgr2=0x7f6f8c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91207700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c108400 0x7f6f8c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f91207700 1 -- 172.21.15.3:0/151806865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c19a070 con 0x7f6f8c102390 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.273+0000 7f6f90a06700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102d50 0x7f6f8c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.274+0000 7f6f91207700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c102390 0x7f6f8c19f750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e9f0 tx=0x7f6f8800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.274+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8800c9c0 con 0x7f6f8c102390 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.274+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c19a360 con 0x7f6f8c102390 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.274+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f8800cb60 con 0x7f6f8c102390 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.274+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88018680 con 0x7f6f8c102390 2024-05-08T05:52:39.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.274+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c1a6620 con 0x7f6f8c102390 2024-05-08T05:52:39.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.275+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c06a190 con 0x7f6f8c102390 2024-05-08T05:52:39.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.277+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6f88010430 con 0x7f6f8c102390 2024-05-08T05:52:39.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.277+0000 7f6f827fc700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6f780b2880 0x7f6f780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.277+0000 7f6f90a06700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6f780b2880 0x7f6f780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.278+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f6f880e2a60 con 0x7f6f8c102390 2024-05-08T05:52:39.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.278+0000 7f6f90a06700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6f780b2880 0x7f6f780b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c100a60 tx=0x7f6f7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:39.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.281+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f880aa390 con 0x7f6f8c102390 2024-05-08T05:52:39.511 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.511+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f6f8c1a6850 con 0x7f6f8c102390 2024-05-08T05:52:39.512 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.512+0000 7f6f827fc700 1 -- 172.21.15.3:0/151806865 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6f880b2410 con 0x7f6f8c102390 2024-05-08T05:52:39.512 INFO:teuthology.orchestra.run.smithi003.stdout:227633266752 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.514+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6f780b2880 msgr2=0x7f6f780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.514+0000 7f6f9346b700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6f780b2880 0x7f6f780b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c100a60 tx=0x7f6f7c00b410 comp rx=0 tx=0).stop 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.514+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c102390 msgr2=0x7f6f8c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.514+0000 7f6f9346b700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c102390 0x7f6f8c19f750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6f8800e9f0 tx=0x7f6f8800ed00 comp rx=0 tx=0).stop 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 shutdown_connections 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f6f780b2880 0x7f6f780b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6f8c108400 0x7f6f8c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f8c102390 0x7f6f8c19f750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 --2- 172.21.15.3:0/151806865 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6f8c102d50 0x7f6f8c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 >> 172.21.15.3:0/151806865 conn(0x7f6f8c079310 msgr2=0x7f6f8c101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 shutdown_connections 2024-05-08T05:52:39.515 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.515+0000 7f6f9346b700 1 -- 172.21.15.3:0/151806865 wait complete. 2024-05-08T05:52:39.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 -- 172.21.15.3:0/781317195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103520 msgr2=0x7fae701039a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 --2- 172.21.15.3:0/781317195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103520 0x7fae701039a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae60009990 tx=0x7fae60009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 -- 172.21.15.3:0/781317195 shutdown_connections 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 --2- 172.21.15.3:0/781317195 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae70103ee0 0x7fae70110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 --2- 172.21.15.3:0/781317195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 0x7fae70109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 --2- 172.21.15.3:0/781317195 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103520 0x7fae701039a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 -- 172.21.15.3:0/781317195 >> 172.21.15.3:0/781317195 conn(0x7fae7007a650 msgr2=0x7fae7007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 -- 172.21.15.3:0/781317195 shutdown_connections 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.517+0000 7fae78763700 1 -- 172.21.15.3:0/781317195 wait complete. 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 Processor -- start 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 -- start start 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae70103520 0x7fae7019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 0x7fae7019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 0x7fae70199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fae701136d0 con 0x7fae70103ee0 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fae70113550 con 0x7fae70103520 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae78763700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fae70113850 con 0x7fae70109530 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.518+0000 7fae75cfe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 0x7fae7019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 0x7fae7019fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51226/0 (socket says 172.21.15.3:51226) 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 -- 172.21.15.3:0/710522181 learned_addr learned my addr 172.21.15.3:0/710522181 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae76d00700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 0x7fae70199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae764ff700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae70103520 0x7fae7019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 -- 172.21.15.3:0/710522181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 msgr2=0x7fae70199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 0x7fae70199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 -- 172.21.15.3:0/710522181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae70103520 msgr2=0x7fae7019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae70103520 0x7fae7019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 -- 172.21.15.3:0/710522181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7019a190 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae76d00700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 0x7fae70199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae75cfe700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 0x7fae7019fdb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae60005910 tx=0x7fae6000b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60027070 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7019a420 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae60022470 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae60017650 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.519+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae701a6740 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.521+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae7006a190 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.523+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fae6002a070 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.524+0000 7fae677fe700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae5c0b2880 0x7fae5c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.524+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fae600e3510 con 0x7fae70103ee0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.524+0000 7fae764ff700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae5c0b2880 0x7fae5c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:39.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.524+0000 7fae764ff700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae5c0b2880 0x7fae5c0b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fae7019ab70 tx=0x7fae6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:39.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.528+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae60013070 con 0x7fae70103ee0 2024-05-08T05:52:39.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.759+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fae7002e510 con 0x7fae70103ee0 2024-05-08T05:52:39.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:39 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1015668926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:52:39.760 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:39 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/151806865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:52:39.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.760+0000 7fae677fe700 1 -- 172.21.15.3:0/710522181 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fae600aaed0 con 0x7fae70103ee0 2024-05-08T05:52:39.761 INFO:teuthology.orchestra.run.smithi003.stdout:77309411422 2024-05-08T05:52:39.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae5c0b2880 msgr2=0x7fae5c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae5c0b2880 0x7fae5c0b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fae7019ab70 tx=0x7fae6c009260 comp rx=0 tx=0).stop 2024-05-08T05:52:39.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 msgr2=0x7fae7019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:39.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 0x7fae7019fdb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae60005910 tx=0x7fae6000b670 comp rx=0 tx=0).stop 2024-05-08T05:52:39.762 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 shutdown_connections 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fae70109530 0x7fae70199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fae70103520 0x7fae7019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fae5c0b2880 0x7fae5c0b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 --2- 172.21.15.3:0/710522181 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fae70103ee0 0x7fae7019fdb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 >> 172.21.15.3:0/710522181 conn(0x7fae7007a650 msgr2=0x7fae701023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 shutdown_connections 2024-05-08T05:52:39.763 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.762+0000 7fae78763700 1 -- 172.21.15.3:0/710522181 wait complete. 2024-05-08T05:52:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:39 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1015668926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:52:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:39 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/151806865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:52:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:39 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1015668926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:52:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:39 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/151806865' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:52:40.318 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266746 got 227633266752 for osd.8 2024-05-08T05:52:40.319 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1458691271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 msgr2=0x7f0f0810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:40.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1458691271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f0810f240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0f0000b0a0 tx=0x7f0f0000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1458691271 shutdown_connections 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1458691271 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f0810f240 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1458691271 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f08108400 0x7f0f081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1458691271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f08102390 0x7f0f08102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1458691271 >> 172.21.15.3:0/1458691271 conn(0x7f0f08079310 msgr2=0x7f0f08079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1458691271 shutdown_connections 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.176+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1458691271 wait complete. 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 Processor -- start 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 -- start start 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f08102390 0x7f0f08104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f081045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f08108400 0x7f0f08104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:40.414 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0f08112520 con 0x7f0f08102d50 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081123a0 con 0x7f0f08108400 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f0d29b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0f081126a0 con 0x7f0f08102390 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f081045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f067fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f081045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51186/0 (socket says 172.21.15.3:51186) 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f067fc700 1 -- 172.21.15.3:0/1015668926 learned_addr learned my addr 172.21.15.3:0/1015668926 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.177+0000 7f0f06ffd700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f08102390 0x7f0f08104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f077fe700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f08108400 0x7f0f08104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f067fc700 1 -- 172.21.15.3:0/1015668926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f08102390 msgr2=0x7f0f08104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f067fc700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f08102390 0x7f0f08104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f067fc700 1 -- 172.21.15.3:0/1015668926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f08108400 msgr2=0x7f0f08104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f067fc700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f08108400 0x7f0f08104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f067fc700 1 -- 172.21.15.3:0/1015668926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f081a7ff0 con 0x7f0f08102d50 2024-05-08T05:52:40.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f067fc700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f081045b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000e9a0 tx=0x7f0ef000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef000cbd0 con 0x7f0f08102d50 2024-05-08T05:52:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef000cd70 con 0x7f0f08102d50 2024-05-08T05:52:40.416 INFO:teuthology.orchestra.run.smithi003.stdout:34359738472 2024-05-08T05:52:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f081a82e0 con 0x7f0f08102d50 2024-05-08T05:52:40.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0018680 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.178+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f081a8820 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.179+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f0806a190 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.181+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f0ef0010430 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.182+0000 7f0efffff700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0ef40b2880 0x7f0ef40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.182+0000 7f0f06ffd700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0ef40b2880 0x7f0ef40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.182+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f0ef00e2d80 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.183+0000 7f0f06ffd700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0ef40b2880 0x7f0ef40b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009be0 tx=0x7f0ef8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.186+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ef00aa6b0 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.403+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0f081a8b40 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.404+0000 7f0efffff700 1 -- 172.21.15.3:0/1015668926 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ef00a9fb0 con 0x7f0f08102d50 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0ef40b2880 msgr2=0x7f0ef40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0ef40b2880 0x7f0ef40b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8009be0 tx=0x7f0ef8009260 comp rx=0 tx=0).stop 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 msgr2=0x7f0f081045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:40.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f081045b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000e9a0 tx=0x7f0ef000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 shutdown_connections 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0f08108400 0x7f0f08104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f08102390 0x7f0f08104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f0ef40b2880 0x7f0ef40b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 --2- 172.21.15.3:0/1015668926 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0f08102d50 0x7f0f081045b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 >> 172.21.15.3:0/1015668926 conn(0x7f0f08079310 msgr2=0x7f0f0806bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.406+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 shutdown_connections 2024-05-08T05:52:40.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:39.407+0000 7f0f0d29b700 1 -- 172.21.15.3:0/1015668926 wait complete. 2024-05-08T05:52:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:40 smithi003 ceph-mon[30137]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:40.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:40 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/710522181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:52:41.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:40 smithi089 ceph-mon[34495]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:41.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:40 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/710522181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:52:41.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:40 smithi104 ceph-mon[34559]: pgmap v382: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:41.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:40 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/710522181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:52:41.086 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738467 got 34359738472 for osd.0 2024-05-08T05:52:41.086 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:41.741 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411414 got 77309411422 for osd.2 2024-05-08T05:52:41.741 DEBUG:teuthology.parallel:result is None 2024-05-08T05:52:41.741 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:52:41.742 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:52:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:52:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:52:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:41 smithi003 ceph-mon[30137]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:41.993 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:52:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:52:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:41 smithi089 ceph-mon[34495]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:52:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:52:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:41 smithi104 ceph-mon[34559]: from='mgr.14150 172.21.15.3:0/614269260' entity='mgr.x' 2024-05-08T05:52:42.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:42 smithi003 ceph-mon[30137]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:43.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:42 smithi089 ceph-mon[34495]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:43.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:42 smithi104 ceph-mon[34559]: pgmap v383: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 -- 172.21.15.3:0/737225912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006c700 msgr2=0x7f8af0105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/737225912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006c700 0x7f8af0105f00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009990 tx=0x7f8ae8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.728 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 -- 172.21.15.3:0/737225912 shutdown_connections 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/737225912 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006c700 0x7f8af0105f00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/737225912 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af01083f0 0x7f8af01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/737225912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8af006bb70 0x7f8af006bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 -- 172.21.15.3:0/737225912 >> 172.21.15.3:0/737225912 conn(0x7f8af0079350 msgr2=0x7f8af0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 -- 172.21.15.3:0/737225912 shutdown_connections 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.726+0000 7f8af7ed2700 1 -- 172.21.15.3:0/737225912 wait complete. 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 Processor -- start 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 -- start start 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 0x7f8af01a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af006c700 0x7f8af01a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8af01083f0 0x7f8af01a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0119d20 con 0x7f8af006bb70 2024-05-08T05:52:43.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0119ba0 con 0x7f8af006c700 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af7ed2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8af0119ea0 con 0x7f8af01083f0 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af5c6e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 0x7f8af01a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af5c6e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 0x7f8af01a0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51922/0 (socket says 172.21.15.3:51922) 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af5c6e700 1 -- 172.21.15.3:0/549920253 learned_addr learned my addr 172.21.15.3:0/549920253 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.727+0000 7f8af546d700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af006c700 0x7f8af01a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af5c6e700 1 -- 172.21.15.3:0/549920253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8af01083f0 msgr2=0x7f8af01a4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af646f700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8af01083f0 0x7f8af01a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af5c6e700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8af01083f0 0x7f8af01a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af5c6e700 1 -- 172.21.15.3:0/549920253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af006c700 msgr2=0x7f8af01a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af5c6e700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af006c700 0x7f8af01a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af5c6e700 1 -- 172.21.15.3:0/549920253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af01a51f0 con 0x7f8af006bb70 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af546d700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af006c700 0x7f8af01a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:52:43.730 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af5c6e700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 0x7f8af01a0230 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00ba50 tx=0x7f8aec00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec019070 con 0x7f8af006bb70 2024-05-08T05:52:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aec014440 con 0x7f8af006bb70 2024-05-08T05:52:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af01a54e0 con 0x7f8af006bb70 2024-05-08T05:52:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0075d0 con 0x7f8af006bb70 2024-05-08T05:52:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.728+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af01acd60 con 0x7f8af006bb70 2024-05-08T05:52:43.731 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.729+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af01029e0 con 0x7f8af006bb70 2024-05-08T05:52:43.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.732+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8aec01c070 con 0x7f8af006bb70 2024-05-08T05:52:43.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.733+0000 7f8ae6ffd700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8adc0b2880 0x7f8adc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:43.733 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.733+0000 7f8af546d700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8adc0b2880 0x7f8adc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:43.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.733+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8aec0e1f60 con 0x7f8af006bb70 2024-05-08T05:52:43.734 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.733+0000 7f8af546d700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8adc0b2880 0x7f8adc0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0005fd0 tx=0x7f8ae0005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:43.735 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.736+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8aec0a9890 con 0x7f8af006bb70 2024-05-08T05:52:43.945 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.944+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8af0101e80 con 0x7f8adc0b2880 2024-05-08T05:52:43.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.954+0000 7f8ae6ffd700 1 -- 172.21.15.3:0/549920253 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340655 (secure 0 0 0) 0x7f8af0101e80 con 0x7f8adc0b2880 2024-05-08T05:52:43.959 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8adc0b2880 msgr2=0x7f8adc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8adc0b2880 0x7f8adc0b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0005fd0 tx=0x7f8ae0005ee0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 msgr2=0x7f8af01a0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 0x7f8af01a0230 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00ba50 tx=0x7f8aec00be10 comp rx=0 tx=0).stop 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 shutdown_connections 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8af01083f0 0x7f8af01a4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8adc0b2880 0x7f8adc0b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8af006bb70 0x7f8af01a0230 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 --2- 172.21.15.3:0/549920253 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8af006c700 0x7f8af01a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:43.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 >> 172.21.15.3:0/549920253 conn(0x7f8af0079350 msgr2=0x7f8af0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:43.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 shutdown_connections 2024-05-08T05:52:43.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:43.958+0000 7f8af7ed2700 1 -- 172.21.15.3:0/549920253 wait complete. 2024-05-08T05:52:43.963 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:52:44.696 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-05-08T05:52:43.257326+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656776,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":289,"num_read_kb":240,"num_write":220,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360980,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580844,"statfs":{"total":1151940427776,"available":1151570784256,"internally_reserved":0,"allocated":12111872,"data_stored":8126138,"data_compressed":495825,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003231"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244785+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:52:33.244785+0000","last_peered":"2024-05-08T05:52:33.244785+0000","last_clean":"2024-05-08T05:52:33.244785+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:52:33.244785+0000","last_undegraded":"2024-05-08T05:52:33.244785+0000","last_fullsized":"2024-05-08T05:52:33.244785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.232262+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:52:33.232262+0000","last_peered":"2024-05-08T05:52:33.232262+0000","last_clean":"2024-05-08T05:52:33.232262+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:52:33.232262+0000","last_undegraded":"2024-05-08T05:52:33.232262+0000","last_fullsized":"2024-05-08T05:52:33.232262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246752+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:52:33.246752+0000","last_peered":"2024-05-08T05:52:33.246752+0000","last_clean":"2024-05-08T05:52:33.246752+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:52:33.246752+0000","last_undegraded":"2024-05-08T05:52:33.246752+0000","last_fullsized":"2024-05-08T05:52:33.246752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.218487+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:52:33.218487+0000","last_peered":"2024-05-08T05:52:33.218487+0000","last_clean":"2024-05-08T05:52:33.218487+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:52:33.218487+0000","last_undegraded":"2024-05-08T05:52:33.218487+0000","last_fullsized":"2024-05-08T05:52:33.218487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.231260+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:52:33.231260+0000","last_peered":"2024-05-08T05:52:33.231260+0000","last_clean":"2024-05-08T05:52:33.231260+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:52:33.231260+0000","last_undegraded":"2024-05-08T05:52:33.231260+0000","last_fullsized":"2024-05-08T05:52:33.231260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.243489+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:52:33.243489+0000","last_peered":"2024-05-08T05:52:33.243489+0000","last_clean":"2024-05-08T05:52:33.243489+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:52:33.243489+0000","last_undegraded":"2024-05-08T05:52:33.243489+0000","last_fullsized":"2024-05-08T05:52:33.243489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240889+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:52:33.240889+0000","last_peered":"2024-05-08T05:52:33.240889+0000","last_clean":"2024-05-08T05:52:33.240889+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:52:33.240889+0000","last_undegraded":"2024-05-08T05:52:33.240889+0000","last_fullsized":"2024-05-08T05:52:33.240889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244620+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:52:33.244620+0000","last_peered":"2024-05-08T05:52:33.244620+0000","last_clean":"2024-05-08T05:52:33.244620+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:52:33.244620+0000","last_undegraded":"2024-05-08T05:52:33.244620+0000","last_fullsized":"2024-05-08T05:52:33.244620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247043+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:52:33.247043+0000","last_peered":"2024-05-08T05:52:33.247043+0000","last_clean":"2024-05-08T05:52:33.247043+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:52:33.247043+0000","last_undegraded":"2024-05-08T05:52:33.247043+0000","last_fullsized":"2024-05-08T05:52:33.247043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.239308+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:52:33.239308+0000","last_peered":"2024-05-08T05:52:33.239308+0000","last_clean":"2024-05-08T05:52:33.239308+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:52:33.239308+0000","last_undegraded":"2024-05-08T05:52:33.239308+0000","last_fullsized":"2024-05-08T05:52:33.239308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244376+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:52:33.244376+0000","last_peered":"2024-05-08T05:52:33.244376+0000","last_clean":"2024-05-08T05:52:33.244376+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:52:33.244376+0000","last_undegraded":"2024-05-08T05:52:33.244376+0000","last_fullsized":"2024-05-08T05:52:33.244376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242093+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:52:33.242093+0000","last_peered":"2024-05-08T05:52:33.242093+0000","last_clean":"2024-05-08T05:52:33.242093+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:52:33.242093+0000","last_undegraded":"2024-05-08T05:52:33.242093+0000","last_fullsized":"2024-05-08T05:52:33.242093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242818+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:52:33.242818+0000","last_peered":"2024-05-08T05:52:33.242818+0000","last_clean":"2024-05-08T05:52:33.242818+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:52:33.242818+0000","last_undegraded":"2024-05-08T05:52:33.242818+0000","last_fullsized":"2024-05-08T05:52:33.242818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245661+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:52:33.245661+0000","last_peered":"2024-05-08T05:52:33.245661+0000","last_clean":"2024-05-08T05:52:33.245661+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:52:33.245661+0000","last_undegraded":"2024-05-08T05:52:33.245661+0000","last_fullsized":"2024-05-08T05:52:33.245661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244111+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:52:33.244111+0000","last_peered":"2024-05-08T05:52:33.244111+0000","last_clean":"2024-05-08T05:52:33.244111+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:52:33.244111+0000","last_undegraded":"2024-05-08T05:52:33.244111+0000","last_fullsized":"2024-05-08T05:52:33.244111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.239721+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:52:33.239721+0000","last_peered":"2024-05-08T05:52:33.239721+0000","last_clean":"2024-05-08T05:52:33.239721+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:52:33.239721+0000","last_undegraded":"2024-05-08T05:52:33.239721+0000","last_fullsized":"2024-05-08T05:52:33.239721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245068+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:52:33.245068+0000","last_peered":"2024-05-08T05:52:33.245068+0000","last_clean":"2024-05-08T05:52:33.245068+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:52:33.245068+0000","last_undegraded":"2024-05-08T05:52:33.245068+0000","last_fullsized":"2024-05-08T05:52:33.245068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242339+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:52:33.242339+0000","last_peered":"2024-05-08T05:52:33.242339+0000","last_clean":"2024-05-08T05:52:33.242339+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:52:33.242339+0000","last_undegraded":"2024-05-08T05:52:33.242339+0000","last_fullsized":"2024-05-08T05:52:33.242339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238286+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:52:33.238286+0000","last_peered":"2024-05-08T05:52:33.238286+0000","last_clean":"2024-05-08T05:52:33.238286+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:52:33.238286+0000","last_undegraded":"2024-05-08T05:52:33.238286+0000","last_fullsized":"2024-05-08T05:52:33.238286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.737387+0000","last_change":"2024-05-08T05:51:35.736506+0000","last_active":"2024-05-08T05:51:35.737387+0000","last_peered":"2024-05-08T05:51:35.737387+0000","last_clean":"2024-05-08T05:51:35.737387+0000","last_became_active":"2024-05-08T05:51:35.735983+0000","last_became_peered":"2024-05-08T05:51:35.735983+0000","last_unstale":"2024-05-08T05:51:35.737387+0000","last_undegraded":"2024-05-08T05:51:35.737387+0000","last_fullsized":"2024-05-08T05:51:35.737387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:57.132249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305752+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:51:35.305752+0000","last_peered":"2024-05-08T05:51:35.305752+0000","last_clean":"2024-05-08T05:51:35.305752+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:51:35.305752+0000","last_undegraded":"2024-05-08T05:51:35.305752+0000","last_fullsized":"2024-05-08T05:51:35.305752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244072+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:52:33.244072+0000","last_peered":"2024-05-08T05:52:33.244072+0000","last_clean":"2024-05-08T05:52:33.244072+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:52:33.244072+0000","last_undegraded":"2024-05-08T05:52:33.244072+0000","last_fullsized":"2024-05-08T05:52:33.244072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734333+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:51:35.734333+0000","last_peered":"2024-05-08T05:51:35.734333+0000","last_clean":"2024-05-08T05:51:35.734333+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:51:35.734333+0000","last_undegraded":"2024-05-08T05:51:35.734333+0000","last_fullsized":"2024-05-08T05:51:35.734333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.559111+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:51:35.559111+0000","last_peered":"2024-05-08T05:51:35.559111+0000","last_clean":"2024-05-08T05:51:35.559111+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:51:35.559111+0000","last_undegraded":"2024-05-08T05:51:35.559111+0000","last_fullsized":"2024-05-08T05:51:35.559111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247314+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:52:33.247314+0000","last_peered":"2024-05-08T05:52:33.247314+0000","last_clean":"2024-05-08T05:52:33.247314+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:52:33.247314+0000","last_undegraded":"2024-05-08T05:52:33.247314+0000","last_fullsized":"2024-05-08T05:52:33.247314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734776+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:51:35.734776+0000","last_peered":"2024-05-08T05:51:35.734776+0000","last_clean":"2024-05-08T05:51:35.734776+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:51:35.734776+0000","last_undegraded":"2024-05-08T05:51:35.734776+0000","last_fullsized":"2024-05-08T05:51:35.734776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735353+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:51:35.735353+0000","last_peered":"2024-05-08T05:51:35.735353+0000","last_clean":"2024-05-08T05:51:35.735353+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:51:35.735353+0000","last_undegraded":"2024-05-08T05:51:35.735353+0000","last_fullsized":"2024-05-08T05:51:35.735353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246126+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:52:33.246126+0000","last_peered":"2024-05-08T05:52:33.246126+0000","last_clean":"2024-05-08T05:52:33.246126+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:52:33.246126+0000","last_undegraded":"2024-05-08T05:52:33.246126+0000","last_fullsized":"2024-05-08T05:52:33.246126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734803+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:51:35.734803+0000","last_peered":"2024-05-08T05:51:35.734803+0000","last_clean":"2024-05-08T05:51:35.734803+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:51:35.734803+0000","last_undegraded":"2024-05-08T05:51:35.734803+0000","last_fullsized":"2024-05-08T05:51:35.734803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.733683+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:51:35.733683+0000","last_peered":"2024-05-08T05:51:35.733683+0000","last_clean":"2024-05-08T05:51:35.733683+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:51:35.733683+0000","last_undegraded":"2024-05-08T05:51:35.733683+0000","last_fullsized":"2024-05-08T05:51:35.733683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247535+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:52:33.247535+0000","last_peered":"2024-05-08T05:52:33.247535+0000","last_clean":"2024-05-08T05:52:33.247535+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:52:33.247535+0000","last_undegraded":"2024-05-08T05:52:33.247535+0000","last_fullsized":"2024-05-08T05:52:33.247535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.737567+0000","last_change":"2024-05-08T05:51:35.736946+0000","last_active":"2024-05-08T05:51:35.737567+0000","last_peered":"2024-05-08T05:51:35.737567+0000","last_clean":"2024-05-08T05:51:35.737567+0000","last_became_active":"2024-05-08T05:51:35.736199+0000","last_became_peered":"2024-05-08T05:51:35.736199+0000","last_unstale":"2024-05-08T05:51:35.737567+0000","last_undegraded":"2024-05-08T05:51:35.737567+0000","last_fullsized":"2024-05-08T05:51:35.737567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:47:09.503650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616666+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:51:35.616666+0000","last_peered":"2024-05-08T05:51:35.616666+0000","last_clean":"2024-05-08T05:51:35.616666+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:51:35.616666+0000","last_undegraded":"2024-05-08T05:51:35.616666+0000","last_fullsized":"2024-05-08T05:51:35.616666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.231014+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:52:33.231014+0000","last_peered":"2024-05-08T05:52:33.231014+0000","last_clean":"2024-05-08T05:52:33.231014+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:52:33.231014+0000","last_undegraded":"2024-05-08T05:52:33.231014+0000","last_fullsized":"2024-05-08T05:52:33.231014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306714+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:51:35.306714+0000","last_peered":"2024-05-08T05:51:35.306714+0000","last_clean":"2024-05-08T05:51:35.306714+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:51:35.306714+0000","last_undegraded":"2024-05-08T05:51:35.306714+0000","last_fullsized":"2024-05-08T05:51:35.306714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.307199+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:51:35.307199+0000","last_peered":"2024-05-08T05:51:35.307199+0000","last_clean":"2024-05-08T05:51:35.307199+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:51:35.307199+0000","last_undegraded":"2024-05-08T05:51:35.307199+0000","last_fullsized":"2024-05-08T05:51:35.307199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242256+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:52:33.242256+0000","last_peered":"2024-05-08T05:52:33.242256+0000","last_clean":"2024-05-08T05:52:33.242256+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:52:33.242256+0000","last_undegraded":"2024-05-08T05:52:33.242256+0000","last_fullsized":"2024-05-08T05:52:33.242256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.733917+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:51:35.733917+0000","last_peered":"2024-05-08T05:51:35.733917+0000","last_clean":"2024-05-08T05:51:35.733917+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:51:35.733917+0000","last_undegraded":"2024-05-08T05:51:35.733917+0000","last_fullsized":"2024-05-08T05:51:35.733917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.699649+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:51:35.699649+0000","last_peered":"2024-05-08T05:51:35.699649+0000","last_clean":"2024-05-08T05:51:35.699649+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:51:35.699649+0000","last_undegraded":"2024-05-08T05:51:35.699649+0000","last_fullsized":"2024-05-08T05:51:35.699649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.241408+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:52:33.241408+0000","last_peered":"2024-05-08T05:52:33.241408+0000","last_clean":"2024-05-08T05:52:33.241408+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:52:33.241408+0000","last_undegraded":"2024-05-08T05:52:33.241408+0000","last_fullsized":"2024-05-08T05:52:33.241408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.732206+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:51:35.732206+0000","last_peered":"2024-05-08T05:51:35.732206+0000","last_clean":"2024-05-08T05:51:35.732206+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:51:35.732206+0000","last_undegraded":"2024-05-08T05:51:35.732206+0000","last_fullsized":"2024-05-08T05:51:35.732206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616203+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:51:35.616203+0000","last_peered":"2024-05-08T05:51:35.616203+0000","last_clean":"2024-05-08T05:51:35.616203+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:51:35.616203+0000","last_undegraded":"2024-05-08T05:51:35.616203+0000","last_fullsized":"2024-05-08T05:51:35.616203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242588+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:52:33.242588+0000","last_peered":"2024-05-08T05:52:33.242588+0000","last_clean":"2024-05-08T05:52:33.242588+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:52:33.242588+0000","last_undegraded":"2024-05-08T05:52:33.242588+0000","last_fullsized":"2024-05-08T05:52:33.242588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.558754+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:51:35.558754+0000","last_peered":"2024-05-08T05:51:35.558754+0000","last_clean":"2024-05-08T05:51:35.558754+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:51:35.558754+0000","last_undegraded":"2024-05-08T05:51:35.558754+0000","last_fullsized":"2024-05-08T05:51:35.558754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.307307+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:51:35.307307+0000","last_peered":"2024-05-08T05:51:35.307307+0000","last_clean":"2024-05-08T05:51:35.307307+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:51:35.307307+0000","last_undegraded":"2024-05-08T05:51:35.307307+0000","last_fullsized":"2024-05-08T05:51:35.307307+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246456+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:52:33.246456+0000","last_peered":"2024-05-08T05:52:33.246456+0000","last_clean":"2024-05-08T05:52:33.246456+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:52:33.246456+0000","last_undegraded":"2024-05-08T05:52:33.246456+0000","last_fullsized":"2024-05-08T05:52:33.246456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305089+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:51:35.305089+0000","last_peered":"2024-05-08T05:51:35.305089+0000","last_clean":"2024-05-08T05:51:35.305089+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:51:35.305089+0000","last_undegraded":"2024-05-08T05:51:35.305089+0000","last_fullsized":"2024-05-08T05:51:35.305089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.481078+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:51:35.481078+0000","last_peered":"2024-05-08T05:51:35.481078+0000","last_clean":"2024-05-08T05:51:35.481078+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:51:35.481078+0000","last_undegraded":"2024-05-08T05:51:35.481078+0000","last_fullsized":"2024-05-08T05:51:35.481078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245974+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:52:33.245974+0000","last_peered":"2024-05-08T05:52:33.245974+0000","last_clean":"2024-05-08T05:52:33.245974+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:52:33.245974+0000","last_undegraded":"2024-05-08T05:52:33.245974+0000","last_fullsized":"2024-05-08T05:52:33.245974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734874+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:51:35.734874+0000","last_peered":"2024-05-08T05:51:35.734874+0000","last_clean":"2024-05-08T05:51:35.734874+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:51:35.734874+0000","last_undegraded":"2024-05-08T05:51:35.734874+0000","last_fullsized":"2024-05-08T05:51:35.734874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616649+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:51:35.616649+0000","last_peered":"2024-05-08T05:51:35.616649+0000","last_clean":"2024-05-08T05:51:35.616649+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:51:35.616649+0000","last_undegraded":"2024-05-08T05:51:35.616649+0000","last_fullsized":"2024-05-08T05:51:35.616649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245970+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:52:33.245970+0000","last_peered":"2024-05-08T05:52:33.245970+0000","last_clean":"2024-05-08T05:52:33.245970+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:52:33.245970+0000","last_undegraded":"2024-05-08T05:52:33.245970+0000","last_fullsized":"2024-05-08T05:52:33.245970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306710+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:51:35.306710+0000","last_peered":"2024-05-08T05:51:35.306710+0000","last_clean":"2024-05-08T05:51:35.306710+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:51:35.306710+0000","last_undegraded":"2024-05-08T05:51:35.306710+0000","last_fullsized":"2024-05-08T05:51:35.306710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":283,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:37.708496+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:52:37.708496+0000","last_peered":"2024-05-08T05:52:37.708496+0000","last_clean":"2024-05-08T05:52:37.708496+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:52:37.708496+0000","last_undegraded":"2024-05-08T05:52:37.708496+0000","last_fullsized":"2024-05-08T05:52:37.708496+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65376,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":202,"num_read_kb":165,"num_write":36,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238558+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:52:33.238558+0000","last_peered":"2024-05-08T05:52:33.238558+0000","last_clean":"2024-05-08T05:52:33.238558+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:52:33.238558+0000","last_undegraded":"2024-05-08T05:52:33.238558+0000","last_fullsized":"2024-05-08T05:52:33.238558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.617056+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:51:35.617056+0000","last_peered":"2024-05-08T05:51:35.617056+0000","last_clean":"2024-05-08T05:51:35.617056+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:51:35.617056+0000","last_undegraded":"2024-05-08T05:51:35.617056+0000","last_fullsized":"2024-05-08T05:51:35.617056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.307011+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:51:35.307011+0000","last_peered":"2024-05-08T05:51:35.307011+0000","last_clean":"2024-05-08T05:51:35.307011+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:51:35.307011+0000","last_undegraded":"2024-05-08T05:51:35.307011+0000","last_fullsized":"2024-05-08T05:51:35.307011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240074+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:52:33.240074+0000","last_peered":"2024-05-08T05:52:33.240074+0000","last_clean":"2024-05-08T05:52:33.240074+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:52:33.240074+0000","last_undegraded":"2024-05-08T05:52:33.240074+0000","last_fullsized":"2024-05-08T05:52:33.240074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.733865+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:51:35.733865+0000","last_peered":"2024-05-08T05:51:35.733865+0000","last_clean":"2024-05-08T05:51:35.733865+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:51:35.733865+0000","last_undegraded":"2024-05-08T05:51:35.733865+0000","last_fullsized":"2024-05-08T05:51:35.733865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616588+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:51:35.616588+0000","last_peered":"2024-05-08T05:51:35.616588+0000","last_clean":"2024-05-08T05:51:35.616588+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:51:35.616588+0000","last_undegraded":"2024-05-08T05:51:35.616588+0000","last_fullsized":"2024-05-08T05:51:35.616588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238566+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:52:33.238566+0000","last_peered":"2024-05-08T05:52:33.238566+0000","last_clean":"2024-05-08T05:52:33.238566+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:52:33.238566+0000","last_undegraded":"2024-05-08T05:52:33.238566+0000","last_fullsized":"2024-05-08T05:52:33.238566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305547+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:51:35.305547+0000","last_peered":"2024-05-08T05:51:35.305547+0000","last_clean":"2024-05-08T05:51:35.305547+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:51:35.305547+0000","last_undegraded":"2024-05-08T05:51:35.305547+0000","last_fullsized":"2024-05-08T05:51:35.305547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735187+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:51:35.735187+0000","last_peered":"2024-05-08T05:51:35.735187+0000","last_clean":"2024-05-08T05:51:35.735187+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:51:35.735187+0000","last_undegraded":"2024-05-08T05:51:35.735187+0000","last_fullsized":"2024-05-08T05:51:35.735187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240768+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:52:33.240768+0000","last_peered":"2024-05-08T05:52:33.240768+0000","last_clean":"2024-05-08T05:52:33.240768+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:52:33.240768+0000","last_undegraded":"2024-05-08T05:52:33.240768+0000","last_fullsized":"2024-05-08T05:52:33.240768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700354+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:51:35.700354+0000","last_peered":"2024-05-08T05:51:35.700354+0000","last_clean":"2024-05-08T05:51:35.700354+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:51:35.700354+0000","last_undegraded":"2024-05-08T05:51:35.700354+0000","last_fullsized":"2024-05-08T05:51:35.700354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735029+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:51:35.735029+0000","last_peered":"2024-05-08T05:51:35.735029+0000","last_clean":"2024-05-08T05:51:35.735029+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:51:35.735029+0000","last_undegraded":"2024-05-08T05:51:35.735029+0000","last_fullsized":"2024-05-08T05:51:35.735029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246530+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:52:33.246530+0000","last_peered":"2024-05-08T05:52:33.246530+0000","last_clean":"2024-05-08T05:52:33.246530+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:52:33.246530+0000","last_undegraded":"2024-05-08T05:52:33.246530+0000","last_fullsized":"2024-05-08T05:52:33.246530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616325+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:51:35.616325+0000","last_peered":"2024-05-08T05:51:35.616325+0000","last_clean":"2024-05-08T05:51:35.616325+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:51:35.616325+0000","last_undegraded":"2024-05-08T05:51:35.616325+0000","last_fullsized":"2024-05-08T05:51:35.616325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735235+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:51:35.735235+0000","last_peered":"2024-05-08T05:51:35.735235+0000","last_clean":"2024-05-08T05:51:35.735235+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:51:35.735235+0000","last_undegraded":"2024-05-08T05:51:35.735235+0000","last_fullsized":"2024-05-08T05:51:35.735235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.222524+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:52:33.222524+0000","last_peered":"2024-05-08T05:52:33.222524+0000","last_clean":"2024-05-08T05:52:33.222524+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:52:33.222524+0000","last_undegraded":"2024-05-08T05:52:33.222524+0000","last_fullsized":"2024-05-08T05:52:33.222524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735313+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:51:35.735313+0000","last_peered":"2024-05-08T05:51:35.735313+0000","last_clean":"2024-05-08T05:51:35.735313+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:51:35.735313+0000","last_undegraded":"2024-05-08T05:51:35.735313+0000","last_fullsized":"2024-05-08T05:51:35.735313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305419+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:51:35.305419+0000","last_peered":"2024-05-08T05:51:35.305419+0000","last_clean":"2024-05-08T05:51:35.305419+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:51:35.305419+0000","last_undegraded":"2024-05-08T05:51:35.305419+0000","last_fullsized":"2024-05-08T05:51:35.305419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.241452+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:52:33.241452+0000","last_peered":"2024-05-08T05:52:33.241452+0000","last_clean":"2024-05-08T05:52:33.241452+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:52:33.241452+0000","last_undegraded":"2024-05-08T05:52:33.241452+0000","last_fullsized":"2024-05-08T05:52:33.241452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700167+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:51:35.700167+0000","last_peered":"2024-05-08T05:51:35.700167+0000","last_clean":"2024-05-08T05:51:35.700167+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:51:35.700167+0000","last_undegraded":"2024-05-08T05:51:35.700167+0000","last_fullsized":"2024-05-08T05:51:35.700167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480772+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:51:35.480772+0000","last_peered":"2024-05-08T05:51:35.480772+0000","last_clean":"2024-05-08T05:51:35.480772+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:51:35.480772+0000","last_undegraded":"2024-05-08T05:51:35.480772+0000","last_fullsized":"2024-05-08T05:51:35.480772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240128+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:52:33.240128+0000","last_peered":"2024-05-08T05:52:33.240128+0000","last_clean":"2024-05-08T05:52:33.240128+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:52:33.240128+0000","last_undegraded":"2024-05-08T05:52:33.240128+0000","last_fullsized":"2024-05-08T05:52:33.240128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700269+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:51:35.700269+0000","last_peered":"2024-05-08T05:51:35.700269+0000","last_clean":"2024-05-08T05:51:35.700269+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:51:35.700269+0000","last_undegraded":"2024-05-08T05:51:35.700269+0000","last_fullsized":"2024-05-08T05:51:35.700269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.237143+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:52:33.237143+0000","last_peered":"2024-05-08T05:52:33.237143+0000","last_clean":"2024-05-08T05:52:33.237143+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:52:33.237143+0000","last_undegraded":"2024-05-08T05:52:33.237143+0000","last_fullsized":"2024-05-08T05:52:33.237143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.699860+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:51:35.699860+0000","last_peered":"2024-05-08T05:51:35.699860+0000","last_clean":"2024-05-08T05:51:35.699860+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:51:35.699860+0000","last_undegraded":"2024-05-08T05:51:35.699860+0000","last_fullsized":"2024-05-08T05:51:35.699860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480986+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:51:35.480986+0000","last_peered":"2024-05-08T05:51:35.480986+0000","last_clean":"2024-05-08T05:51:35.480986+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:51:35.480986+0000","last_undegraded":"2024-05-08T05:51:35.480986+0000","last_fullsized":"2024-05-08T05:51:35.480986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247419+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:52:33.247419+0000","last_peered":"2024-05-08T05:52:33.247419+0000","last_clean":"2024-05-08T05:52:33.247419+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:52:33.247419+0000","last_undegraded":"2024-05-08T05:52:33.247419+0000","last_fullsized":"2024-05-08T05:52:33.247419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734647+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:51:35.734647+0000","last_peered":"2024-05-08T05:51:35.734647+0000","last_clean":"2024-05-08T05:51:35.734647+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:51:35.734647+0000","last_undegraded":"2024-05-08T05:51:35.734647+0000","last_fullsized":"2024-05-08T05:51:35.734647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480363+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:51:35.480363+0000","last_peered":"2024-05-08T05:51:35.480363+0000","last_clean":"2024-05-08T05:51:35.480363+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:51:35.480363+0000","last_undegraded":"2024-05-08T05:51:35.480363+0000","last_fullsized":"2024-05-08T05:51:35.480363+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616764+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:51:35.616764+0000","last_peered":"2024-05-08T05:51:35.616764+0000","last_clean":"2024-05-08T05:51:35.616764+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:51:35.616764+0000","last_undegraded":"2024-05-08T05:51:35.616764+0000","last_fullsized":"2024-05-08T05:51:35.616764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.222563+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:52:33.222563+0000","last_peered":"2024-05-08T05:52:33.222563+0000","last_clean":"2024-05-08T05:52:33.222563+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:52:33.222563+0000","last_undegraded":"2024-05-08T05:52:33.222563+0000","last_fullsized":"2024-05-08T05:52:33.222563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480616+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:51:35.480616+0000","last_peered":"2024-05-08T05:51:35.480616+0000","last_clean":"2024-05-08T05:51:35.480616+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:51:35.480616+0000","last_undegraded":"2024-05-08T05:51:35.480616+0000","last_fullsized":"2024-05-08T05:51:35.480616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734737+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:51:35.734737+0000","last_peered":"2024-05-08T05:51:35.734737+0000","last_clean":"2024-05-08T05:51:35.734737+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:51:35.734737+0000","last_undegraded":"2024-05-08T05:51:35.734737+0000","last_fullsized":"2024-05-08T05:51:35.734737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735565+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:51:35.735565+0000","last_peered":"2024-05-08T05:51:35.735565+0000","last_clean":"2024-05-08T05:51:35.735565+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:51:35.735565+0000","last_undegraded":"2024-05-08T05:51:35.735565+0000","last_fullsized":"2024-05-08T05:51:35.735565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.231031+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:52:33.231031+0000","last_peered":"2024-05-08T05:52:33.231031+0000","last_clean":"2024-05-08T05:52:33.231031+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:52:33.231031+0000","last_undegraded":"2024-05-08T05:52:33.231031+0000","last_fullsized":"2024-05-08T05:52:33.231031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.558480+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:51:35.558480+0000","last_peered":"2024-05-08T05:51:35.558480+0000","last_clean":"2024-05-08T05:51:35.558480+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:51:35.558480+0000","last_undegraded":"2024-05-08T05:51:35.558480+0000","last_fullsized":"2024-05-08T05:51:35.558480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:06.096275+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:52:06.096275+0000","last_peered":"2024-05-08T05:52:06.096275+0000","last_clean":"2024-05-08T05:52:06.096275+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:52:06.096275+0000","last_undegraded":"2024-05-08T05:52:06.096275+0000","last_fullsized":"2024-05-08T05:52:06.096275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240434+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:52:33.240434+0000","last_peered":"2024-05-08T05:52:33.240434+0000","last_clean":"2024-05-08T05:52:33.240434+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:52:33.240434+0000","last_undegraded":"2024-05-08T05:52:33.240434+0000","last_fullsized":"2024-05-08T05:52:33.240434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306587+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:51:35.306587+0000","last_peered":"2024-05-08T05:51:35.306587+0000","last_clean":"2024-05-08T05:51:35.306587+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:51:35.306587+0000","last_undegraded":"2024-05-08T05:51:35.306587+0000","last_fullsized":"2024-05-08T05:51:35.306587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735111+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:51:35.735111+0000","last_peered":"2024-05-08T05:51:35.735111+0000","last_clean":"2024-05-08T05:51:35.735111+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:51:35.735111+0000","last_undegraded":"2024-05-08T05:51:35.735111+0000","last_fullsized":"2024-05-08T05:51:35.735111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.236016+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:52:33.236016+0000","last_peered":"2024-05-08T05:52:33.236016+0000","last_clean":"2024-05-08T05:52:33.236016+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:52:33.236016+0000","last_undegraded":"2024-05-08T05:52:33.236016+0000","last_fullsized":"2024-05-08T05:52:33.236016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734828+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:51:35.734828+0000","last_peered":"2024-05-08T05:51:35.734828+0000","last_clean":"2024-05-08T05:51:35.734828+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:51:35.734828+0000","last_undegraded":"2024-05-08T05:51:35.734828+0000","last_fullsized":"2024-05-08T05:51:35.734828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734814+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:51:35.734814+0000","last_peered":"2024-05-08T05:51:35.734814+0000","last_clean":"2024-05-08T05:51:35.734814+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:51:35.734814+0000","last_undegraded":"2024-05-08T05:51:35.734814+0000","last_fullsized":"2024-05-08T05:51:35.734814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.220317+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:52:33.220317+0000","last_peered":"2024-05-08T05:52:33.220317+0000","last_clean":"2024-05-08T05:52:33.220317+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:52:33.220317+0000","last_undegraded":"2024-05-08T05:52:33.220317+0000","last_fullsized":"2024-05-08T05:52:33.220317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305660+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:51:35.305660+0000","last_peered":"2024-05-08T05:51:35.305660+0000","last_clean":"2024-05-08T05:51:35.305660+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:51:35.305660+0000","last_undegraded":"2024-05-08T05:51:35.305660+0000","last_fullsized":"2024-05-08T05:51:35.305660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.558899+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:51:35.558899+0000","last_peered":"2024-05-08T05:51:35.558899+0000","last_clean":"2024-05-08T05:51:35.558899+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:51:35.558899+0000","last_undegraded":"2024-05-08T05:51:35.558899+0000","last_fullsized":"2024-05-08T05:51:35.558899+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.237013+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:52:33.237013+0000","last_peered":"2024-05-08T05:52:33.237013+0000","last_clean":"2024-05-08T05:52:33.237013+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:52:33.237013+0000","last_undegraded":"2024-05-08T05:52:33.237013+0000","last_fullsized":"2024-05-08T05:52:33.237013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734948+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:51:35.734948+0000","last_peered":"2024-05-08T05:51:35.734948+0000","last_clean":"2024-05-08T05:51:35.734948+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:51:35.734948+0000","last_undegraded":"2024-05-08T05:51:35.734948+0000","last_fullsized":"2024-05-08T05:51:35.734948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306747+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:51:35.306747+0000","last_peered":"2024-05-08T05:51:35.306747+0000","last_clean":"2024-05-08T05:51:35.306747+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:51:35.306747+0000","last_undegraded":"2024-05-08T05:51:35.306747+0000","last_fullsized":"2024-05-08T05:51:35.306747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.236615+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:52:33.236615+0000","last_peered":"2024-05-08T05:52:33.236615+0000","last_clean":"2024-05-08T05:52:33.236615+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:52:33.236615+0000","last_undegraded":"2024-05-08T05:52:33.236615+0000","last_fullsized":"2024-05-08T05:52:33.236615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734602+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:51:35.734602+0000","last_peered":"2024-05-08T05:51:35.734602+0000","last_clean":"2024-05-08T05:51:35.734602+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:51:35.734602+0000","last_undegraded":"2024-05-08T05:51:35.734602+0000","last_fullsized":"2024-05-08T05:51:35.734602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735276+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:51:35.735276+0000","last_peered":"2024-05-08T05:51:35.735276+0000","last_clean":"2024-05-08T05:51:35.735276+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:51:35.735276+0000","last_undegraded":"2024-05-08T05:51:35.735276+0000","last_fullsized":"2024-05-08T05:51:35.735276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244107+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:52:33.244107+0000","last_peered":"2024-05-08T05:52:33.244107+0000","last_clean":"2024-05-08T05:52:33.244107+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:52:33.244107+0000","last_undegraded":"2024-05-08T05:52:33.244107+0000","last_fullsized":"2024-05-08T05:52:33.244107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735740+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:51:35.735740+0000","last_peered":"2024-05-08T05:51:35.735740+0000","last_clean":"2024-05-08T05:51:35.735740+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:51:35.735740+0000","last_undegraded":"2024-05-08T05:51:35.735740+0000","last_fullsized":"2024-05-08T05:51:35.735740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306985+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:51:35.306985+0000","last_peered":"2024-05-08T05:51:35.306985+0000","last_clean":"2024-05-08T05:51:35.306985+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:51:35.306985+0000","last_undegraded":"2024-05-08T05:51:35.306985+0000","last_fullsized":"2024-05-08T05:51:35.306985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244500+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:52:33.244500+0000","last_peered":"2024-05-08T05:52:33.244500+0000","last_clean":"2024-05-08T05:52:33.244500+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:52:33.244500+0000","last_undegraded":"2024-05-08T05:52:33.244500+0000","last_fullsized":"2024-05-08T05:52:33.244500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.731449+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:51:35.731449+0000","last_peered":"2024-05-08T05:51:35.731449+0000","last_clean":"2024-05-08T05:51:35.731449+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:51:35.731449+0000","last_undegraded":"2024-05-08T05:51:35.731449+0000","last_fullsized":"2024-05-08T05:51:35.731449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616933+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:51:35.616933+0000","last_peered":"2024-05-08T05:51:35.616933+0000","last_clean":"2024-05-08T05:51:35.616933+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:51:35.616933+0000","last_undegraded":"2024-05-08T05:51:35.616933+0000","last_fullsized":"2024-05-08T05:51:35.616933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240971+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:52:33.240971+0000","last_peered":"2024-05-08T05:52:33.240971+0000","last_clean":"2024-05-08T05:52:33.240971+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:52:33.240971+0000","last_undegraded":"2024-05-08T05:52:33.240971+0000","last_fullsized":"2024-05-08T05:52:33.240971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616156+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:51:35.616156+0000","last_peered":"2024-05-08T05:51:35.616156+0000","last_clean":"2024-05-08T05:51:35.616156+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:51:35.616156+0000","last_undegraded":"2024-05-08T05:51:35.616156+0000","last_fullsized":"2024-05-08T05:51:35.616156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700459+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:51:35.700459+0000","last_peered":"2024-05-08T05:51:35.700459+0000","last_clean":"2024-05-08T05:51:35.700459+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:51:35.700459+0000","last_undegraded":"2024-05-08T05:51:35.700459+0000","last_fullsized":"2024-05-08T05:51:35.700459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238584+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:52:33.238584+0000","last_peered":"2024-05-08T05:52:33.238584+0000","last_clean":"2024-05-08T05:52:33.238584+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:52:33.238584+0000","last_undegraded":"2024-05-08T05:52:33.238584+0000","last_fullsized":"2024-05-08T05:52:33.238584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238497+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:52:33.238497+0000","last_peered":"2024-05-08T05:52:33.238497+0000","last_clean":"2024-05-08T05:52:33.238497+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:52:33.238497+0000","last_undegraded":"2024-05-08T05:52:33.238497+0000","last_fullsized":"2024-05-08T05:52:33.238497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.241912+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:52:33.241912+0000","last_peered":"2024-05-08T05:52:33.241912+0000","last_clean":"2024-05-08T05:52:33.241912+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:52:33.241912+0000","last_undegraded":"2024-05-08T05:52:33.241912+0000","last_fullsized":"2024-05-08T05:52:33.241912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245814+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:52:33.245814+0000","last_peered":"2024-05-08T05:52:33.245814+0000","last_clean":"2024-05-08T05:52:33.245814+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:52:33.245814+0000","last_undegraded":"2024-05-08T05:52:33.245814+0000","last_fullsized":"2024-05-08T05:52:33.245814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245127+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:52:33.245127+0000","last_peered":"2024-05-08T05:52:33.245127+0000","last_clean":"2024-05-08T05:52:33.245127+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:52:33.245127+0000","last_undegraded":"2024-05-08T05:52:33.245127+0000","last_fullsized":"2024-05-08T05:52:33.245127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244876+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:52:33.244876+0000","last_peered":"2024-05-08T05:52:33.244876+0000","last_clean":"2024-05-08T05:52:33.244876+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:52:33.244876+0000","last_undegraded":"2024-05-08T05:52:33.244876+0000","last_fullsized":"2024-05-08T05:52:33.244876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.223223+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:52:33.223223+0000","last_peered":"2024-05-08T05:52:33.223223+0000","last_clean":"2024-05-08T05:52:33.223223+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:52:33.223223+0000","last_undegraded":"2024-05-08T05:52:33.223223+0000","last_fullsized":"2024-05-08T05:52:33.223223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246828+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:52:33.246828+0000","last_peered":"2024-05-08T05:52:33.246828+0000","last_clean":"2024-05-08T05:52:33.246828+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:52:33.246828+0000","last_undegraded":"2024-05-08T05:52:33.246828+0000","last_fullsized":"2024-05-08T05:52:33.246828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.236084+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:52:33.236084+0000","last_peered":"2024-05-08T05:52:33.236084+0000","last_clean":"2024-05-08T05:52:33.236084+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:52:33.236084+0000","last_undegraded":"2024-05-08T05:52:33.236084+0000","last_fullsized":"2024-05-08T05:52:33.236084+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242312+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:52:33.242312+0000","last_peered":"2024-05-08T05:52:33.242312+0000","last_clean":"2024-05-08T05:52:33.242312+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:52:33.242312+0000","last_undegraded":"2024-05-08T05:52:33.242312+0000","last_fullsized":"2024-05-08T05:52:33.242312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.222572+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:52:33.222572+0000","last_peered":"2024-05-08T05:52:33.222572+0000","last_clean":"2024-05-08T05:52:33.222572+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:52:33.222572+0000","last_undegraded":"2024-05-08T05:52:33.222572+0000","last_fullsized":"2024-05-08T05:52:33.222572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246246+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:52:33.246246+0000","last_peered":"2024-05-08T05:52:33.246246+0000","last_clean":"2024-05-08T05:52:33.246246+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:52:33.246246+0000","last_undegraded":"2024-05-08T05:52:33.246246+0000","last_fullsized":"2024-05-08T05:52:33.246246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246412+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:52:33.246412+0000","last_peered":"2024-05-08T05:52:33.246412+0000","last_clean":"2024-05-08T05:52:33.246412+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:52:33.246412+0000","last_undegraded":"2024-05-08T05:52:33.246412+0000","last_fullsized":"2024-05-08T05:52:33.246412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.234119+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:52:33.234119+0000","last_peered":"2024-05-08T05:52:33.234119+0000","last_clean":"2024-05-08T05:52:33.234119+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:52:33.234119+0000","last_undegraded":"2024-05-08T05:52:33.234119+0000","last_fullsized":"2024-05-08T05:52:33.234119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66174,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":208,"num_read_kb":171,"num_write":95,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":198522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710768,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":514408,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906997,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1166250,"data_compressed":68051,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070524,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":519086,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266753,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":515012,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462981,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1109454,"data_compressed":68051,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626507,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":510312,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":575688,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":510402,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":575722,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":510416,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986262,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":514430,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822736,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1104958,"data_compressed":68051,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:52:44.702 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:52:44.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:44 smithi003 ceph-mon[30137]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:44.954 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:45.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:44 smithi089 ceph-mon[34495]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:45.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:44 smithi104 ceph-mon[34559]: pgmap v384: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:45 smithi089 ceph-mon[34495]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:46.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:45 smithi104 ceph-mon[34559]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:46.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:45 smithi003 ceph-mon[30137]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 -- 172.21.15.3:0/3384387368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c104130 msgr2=0x7fe39c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 --2- 172.21.15.3:0/3384387368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c104130 0x7fe39c108140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe3840099c0 tx=0x7fe384009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 -- 172.21.15.3:0/3384387368 shutdown_connections 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 --2- 172.21.15.3:0/3384387368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe39c108870 0x7fe39c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 --2- 172.21.15.3:0/3384387368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c103780 0x7fe39c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 --2- 172.21.15.3:0/3384387368 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c104130 0x7fe39c108140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 -- 172.21.15.3:0/3384387368 >> 172.21.15.3:0/3384387368 conn(0x7fe39c07a610 msgr2=0x7fe39c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 -- 172.21.15.3:0/3384387368 shutdown_connections 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.732+0000 7fe3a2815700 1 -- 172.21.15.3:0/3384387368 wait complete. 2024-05-08T05:52:46.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 Processor -- start 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 -- start start 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c103780 0x7fe39c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe39c104130 0x7fe39c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 0x7fe39c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c113630 con 0x7fe39c108870 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c1134b0 con 0x7fe39c103780 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a2815700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c1137b0 con 0x7fe39c104130 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a0db2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 0x7fe39c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a0db2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 0x7fe39c076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51940/0 (socket says 172.21.15.3:51940) 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe3a0db2700 1 -- 172.21.15.3:0/1257561943 learned_addr learned my addr 172.21.15.3:0/1257561943 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe39bfff700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c103780 0x7fe39c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.733+0000 7fe393fff700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe39c104130 0x7fe39c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a0db2700 1 -- 172.21.15.3:0/1257561943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe39c104130 msgr2=0x7fe39c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a0db2700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe39c104130 0x7fe39c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a0db2700 1 -- 172.21.15.3:0/1257561943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c103780 msgr2=0x7fe39c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a0db2700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c103780 0x7fe39c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a0db2700 1 -- 172.21.15.3:0/1257561943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c1a7f40 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a0db2700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 0x7fe39c076220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe394007d70 tx=0x7fe39400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394016020 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39400ac90 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c1a8230 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3940115e0 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.734+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a86f0 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.735+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c06a190 con 0x7fe39c108870 2024-05-08T05:52:46.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.736+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fe394019020 con 0x7fe39c108870 2024-05-08T05:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.738+0000 7fe399ffb700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe3880b2880 0x7fe3880b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.738+0000 7fe39bfff700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe3880b2880 0x7fe3880b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.738+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fe3940e29e0 con 0x7fe39c108870 2024-05-08T05:52:46.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.738+0000 7fe39bfff700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe3880b2880 0x7fe3880b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00ac50 tx=0x7fe38c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:46.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.741+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe394020020 con 0x7fe39c108870 2024-05-08T05:52:46.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:46 smithi003 ceph-mon[30137]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:46.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.952+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe39c02e620 con 0x7fe3880b2880 2024-05-08T05:52:46.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.962+0000 7fe399ffb700 1 -- 172.21.15.3:0/1257561943 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340655 (secure 0 0 0) 0x7fe39c02e620 con 0x7fe3880b2880 2024-05-08T05:52:46.967 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:52:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe3880b2880 msgr2=0x7fe3880b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe3880b2880 0x7fe3880b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00ac50 tx=0x7fe38c00a380 comp rx=0 tx=0).stop 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 msgr2=0x7fe39c076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 0x7fe39c076220 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe394007d70 tx=0x7fe39400a340 comp rx=0 tx=0).stop 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 shutdown_connections 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fe3880b2880 0x7fe3880b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe39c108870 0x7fe39c076220 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe39c103780 0x7fe39c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 --2- 172.21.15.3:0/1257561943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe39c104130 0x7fe39c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 >> 172.21.15.3:0/1257561943 conn(0x7fe39c07a610 msgr2=0x7fe39c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 shutdown_connections 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:46.965+0000 7fe3a2815700 1 -- 172.21.15.3:0/1257561943 wait complete. 2024-05-08T05:52:46.970 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:52:47.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:46 smithi089 ceph-mon[34495]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:47.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:46 smithi104 ceph-mon[34559]: pgmap v385: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:52:47.774 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-05-08T05:52:45.257915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656776,"num_objects":26,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":299,"num_read_kb":245,"num_write":220,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360980,"kb_used_data":11828,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580844,"statfs":{"total":1151940427776,"available":1151570784256,"internally_reserved":0,"allocated":12111872,"data_stored":8126138,"data_compressed":495825,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":16,"num_write":5,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003300"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244785+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:52:33.244785+0000","last_peered":"2024-05-08T05:52:33.244785+0000","last_clean":"2024-05-08T05:52:33.244785+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:52:33.244785+0000","last_undegraded":"2024-05-08T05:52:33.244785+0000","last_fullsized":"2024-05-08T05:52:33.244785+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.232262+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:52:33.232262+0000","last_peered":"2024-05-08T05:52:33.232262+0000","last_clean":"2024-05-08T05:52:33.232262+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:52:33.232262+0000","last_undegraded":"2024-05-08T05:52:33.232262+0000","last_fullsized":"2024-05-08T05:52:33.232262+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246752+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:52:33.246752+0000","last_peered":"2024-05-08T05:52:33.246752+0000","last_clean":"2024-05-08T05:52:33.246752+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:52:33.246752+0000","last_undegraded":"2024-05-08T05:52:33.246752+0000","last_fullsized":"2024-05-08T05:52:33.246752+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.218487+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:52:33.218487+0000","last_peered":"2024-05-08T05:52:33.218487+0000","last_clean":"2024-05-08T05:52:33.218487+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:52:33.218487+0000","last_undegraded":"2024-05-08T05:52:33.218487+0000","last_fullsized":"2024-05-08T05:52:33.218487+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.231260+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:52:33.231260+0000","last_peered":"2024-05-08T05:52:33.231260+0000","last_clean":"2024-05-08T05:52:33.231260+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:52:33.231260+0000","last_undegraded":"2024-05-08T05:52:33.231260+0000","last_fullsized":"2024-05-08T05:52:33.231260+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.243489+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:52:33.243489+0000","last_peered":"2024-05-08T05:52:33.243489+0000","last_clean":"2024-05-08T05:52:33.243489+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:52:33.243489+0000","last_undegraded":"2024-05-08T05:52:33.243489+0000","last_fullsized":"2024-05-08T05:52:33.243489+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240889+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:52:33.240889+0000","last_peered":"2024-05-08T05:52:33.240889+0000","last_clean":"2024-05-08T05:52:33.240889+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:52:33.240889+0000","last_undegraded":"2024-05-08T05:52:33.240889+0000","last_fullsized":"2024-05-08T05:52:33.240889+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244620+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:52:33.244620+0000","last_peered":"2024-05-08T05:52:33.244620+0000","last_clean":"2024-05-08T05:52:33.244620+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:52:33.244620+0000","last_undegraded":"2024-05-08T05:52:33.244620+0000","last_fullsized":"2024-05-08T05:52:33.244620+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247043+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:52:33.247043+0000","last_peered":"2024-05-08T05:52:33.247043+0000","last_clean":"2024-05-08T05:52:33.247043+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:52:33.247043+0000","last_undegraded":"2024-05-08T05:52:33.247043+0000","last_fullsized":"2024-05-08T05:52:33.247043+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":55,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.239308+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:52:33.239308+0000","last_peered":"2024-05-08T05:52:33.239308+0000","last_clean":"2024-05-08T05:52:33.239308+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:52:33.239308+0000","last_undegraded":"2024-05-08T05:52:33.239308+0000","last_fullsized":"2024-05-08T05:52:33.239308+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244376+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:52:33.244376+0000","last_peered":"2024-05-08T05:52:33.244376+0000","last_clean":"2024-05-08T05:52:33.244376+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:52:33.244376+0000","last_undegraded":"2024-05-08T05:52:33.244376+0000","last_fullsized":"2024-05-08T05:52:33.244376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242093+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:52:33.242093+0000","last_peered":"2024-05-08T05:52:33.242093+0000","last_clean":"2024-05-08T05:52:33.242093+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:52:33.242093+0000","last_undegraded":"2024-05-08T05:52:33.242093+0000","last_fullsized":"2024-05-08T05:52:33.242093+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242818+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:52:33.242818+0000","last_peered":"2024-05-08T05:52:33.242818+0000","last_clean":"2024-05-08T05:52:33.242818+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:52:33.242818+0000","last_undegraded":"2024-05-08T05:52:33.242818+0000","last_fullsized":"2024-05-08T05:52:33.242818+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245661+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:52:33.245661+0000","last_peered":"2024-05-08T05:52:33.245661+0000","last_clean":"2024-05-08T05:52:33.245661+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:52:33.245661+0000","last_undegraded":"2024-05-08T05:52:33.245661+0000","last_fullsized":"2024-05-08T05:52:33.245661+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244111+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:52:33.244111+0000","last_peered":"2024-05-08T05:52:33.244111+0000","last_clean":"2024-05-08T05:52:33.244111+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:52:33.244111+0000","last_undegraded":"2024-05-08T05:52:33.244111+0000","last_fullsized":"2024-05-08T05:52:33.244111+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.239721+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:52:33.239721+0000","last_peered":"2024-05-08T05:52:33.239721+0000","last_clean":"2024-05-08T05:52:33.239721+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:52:33.239721+0000","last_undegraded":"2024-05-08T05:52:33.239721+0000","last_fullsized":"2024-05-08T05:52:33.239721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245068+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:52:33.245068+0000","last_peered":"2024-05-08T05:52:33.245068+0000","last_clean":"2024-05-08T05:52:33.245068+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:52:33.245068+0000","last_undegraded":"2024-05-08T05:52:33.245068+0000","last_fullsized":"2024-05-08T05:52:33.245068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242339+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:52:33.242339+0000","last_peered":"2024-05-08T05:52:33.242339+0000","last_clean":"2024-05-08T05:52:33.242339+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:52:33.242339+0000","last_undegraded":"2024-05-08T05:52:33.242339+0000","last_fullsized":"2024-05-08T05:52:33.242339+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238286+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:52:33.238286+0000","last_peered":"2024-05-08T05:52:33.238286+0000","last_clean":"2024-05-08T05:52:33.238286+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:52:33.238286+0000","last_undegraded":"2024-05-08T05:52:33.238286+0000","last_fullsized":"2024-05-08T05:52:33.238286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.737387+0000","last_change":"2024-05-08T05:51:35.736506+0000","last_active":"2024-05-08T05:51:35.737387+0000","last_peered":"2024-05-08T05:51:35.737387+0000","last_clean":"2024-05-08T05:51:35.737387+0000","last_became_active":"2024-05-08T05:51:35.735983+0000","last_became_peered":"2024-05-08T05:51:35.735983+0000","last_unstale":"2024-05-08T05:51:35.737387+0000","last_undegraded":"2024-05-08T05:51:35.737387+0000","last_fullsized":"2024-05-08T05:51:35.737387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:57.132249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305752+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:51:35.305752+0000","last_peered":"2024-05-08T05:51:35.305752+0000","last_clean":"2024-05-08T05:51:35.305752+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:51:35.305752+0000","last_undegraded":"2024-05-08T05:51:35.305752+0000","last_fullsized":"2024-05-08T05:51:35.305752+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244072+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:52:33.244072+0000","last_peered":"2024-05-08T05:52:33.244072+0000","last_clean":"2024-05-08T05:52:33.244072+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:52:33.244072+0000","last_undegraded":"2024-05-08T05:52:33.244072+0000","last_fullsized":"2024-05-08T05:52:33.244072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734333+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:51:35.734333+0000","last_peered":"2024-05-08T05:51:35.734333+0000","last_clean":"2024-05-08T05:51:35.734333+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:51:35.734333+0000","last_undegraded":"2024-05-08T05:51:35.734333+0000","last_fullsized":"2024-05-08T05:51:35.734333+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.559111+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:51:35.559111+0000","last_peered":"2024-05-08T05:51:35.559111+0000","last_clean":"2024-05-08T05:51:35.559111+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:51:35.559111+0000","last_undegraded":"2024-05-08T05:51:35.559111+0000","last_fullsized":"2024-05-08T05:51:35.559111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247314+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:52:33.247314+0000","last_peered":"2024-05-08T05:52:33.247314+0000","last_clean":"2024-05-08T05:52:33.247314+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:52:33.247314+0000","last_undegraded":"2024-05-08T05:52:33.247314+0000","last_fullsized":"2024-05-08T05:52:33.247314+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734776+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:51:35.734776+0000","last_peered":"2024-05-08T05:51:35.734776+0000","last_clean":"2024-05-08T05:51:35.734776+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:51:35.734776+0000","last_undegraded":"2024-05-08T05:51:35.734776+0000","last_fullsized":"2024-05-08T05:51:35.734776+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735353+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:51:35.735353+0000","last_peered":"2024-05-08T05:51:35.735353+0000","last_clean":"2024-05-08T05:51:35.735353+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:51:35.735353+0000","last_undegraded":"2024-05-08T05:51:35.735353+0000","last_fullsized":"2024-05-08T05:51:35.735353+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246126+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:52:33.246126+0000","last_peered":"2024-05-08T05:52:33.246126+0000","last_clean":"2024-05-08T05:52:33.246126+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:52:33.246126+0000","last_undegraded":"2024-05-08T05:52:33.246126+0000","last_fullsized":"2024-05-08T05:52:33.246126+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734803+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:51:35.734803+0000","last_peered":"2024-05-08T05:51:35.734803+0000","last_clean":"2024-05-08T05:51:35.734803+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:51:35.734803+0000","last_undegraded":"2024-05-08T05:51:35.734803+0000","last_fullsized":"2024-05-08T05:51:35.734803+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":19,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.733683+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:51:35.733683+0000","last_peered":"2024-05-08T05:51:35.733683+0000","last_clean":"2024-05-08T05:51:35.733683+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:51:35.733683+0000","last_undegraded":"2024-05-08T05:51:35.733683+0000","last_fullsized":"2024-05-08T05:51:35.733683+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247535+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:52:33.247535+0000","last_peered":"2024-05-08T05:52:33.247535+0000","last_clean":"2024-05-08T05:52:33.247535+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:52:33.247535+0000","last_undegraded":"2024-05-08T05:52:33.247535+0000","last_fullsized":"2024-05-08T05:52:33.247535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.737567+0000","last_change":"2024-05-08T05:51:35.736946+0000","last_active":"2024-05-08T05:51:35.737567+0000","last_peered":"2024-05-08T05:51:35.737567+0000","last_clean":"2024-05-08T05:51:35.737567+0000","last_became_active":"2024-05-08T05:51:35.736199+0000","last_became_peered":"2024-05-08T05:51:35.736199+0000","last_unstale":"2024-05-08T05:51:35.737567+0000","last_undegraded":"2024-05-08T05:51:35.737567+0000","last_fullsized":"2024-05-08T05:51:35.737567+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:47:09.503650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616666+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:51:35.616666+0000","last_peered":"2024-05-08T05:51:35.616666+0000","last_clean":"2024-05-08T05:51:35.616666+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:51:35.616666+0000","last_undegraded":"2024-05-08T05:51:35.616666+0000","last_fullsized":"2024-05-08T05:51:35.616666+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.231014+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:52:33.231014+0000","last_peered":"2024-05-08T05:52:33.231014+0000","last_clean":"2024-05-08T05:52:33.231014+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:52:33.231014+0000","last_undegraded":"2024-05-08T05:52:33.231014+0000","last_fullsized":"2024-05-08T05:52:33.231014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306714+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:51:35.306714+0000","last_peered":"2024-05-08T05:51:35.306714+0000","last_clean":"2024-05-08T05:51:35.306714+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:51:35.306714+0000","last_undegraded":"2024-05-08T05:51:35.306714+0000","last_fullsized":"2024-05-08T05:51:35.306714+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.307199+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:51:35.307199+0000","last_peered":"2024-05-08T05:51:35.307199+0000","last_clean":"2024-05-08T05:51:35.307199+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:51:35.307199+0000","last_undegraded":"2024-05-08T05:51:35.307199+0000","last_fullsized":"2024-05-08T05:51:35.307199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242256+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:52:33.242256+0000","last_peered":"2024-05-08T05:52:33.242256+0000","last_clean":"2024-05-08T05:52:33.242256+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:52:33.242256+0000","last_undegraded":"2024-05-08T05:52:33.242256+0000","last_fullsized":"2024-05-08T05:52:33.242256+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.733917+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:51:35.733917+0000","last_peered":"2024-05-08T05:51:35.733917+0000","last_clean":"2024-05-08T05:51:35.733917+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:51:35.733917+0000","last_undegraded":"2024-05-08T05:51:35.733917+0000","last_fullsized":"2024-05-08T05:51:35.733917+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.699649+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:51:35.699649+0000","last_peered":"2024-05-08T05:51:35.699649+0000","last_clean":"2024-05-08T05:51:35.699649+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:51:35.699649+0000","last_undegraded":"2024-05-08T05:51:35.699649+0000","last_fullsized":"2024-05-08T05:51:35.699649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.241408+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:52:33.241408+0000","last_peered":"2024-05-08T05:52:33.241408+0000","last_clean":"2024-05-08T05:52:33.241408+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:52:33.241408+0000","last_undegraded":"2024-05-08T05:52:33.241408+0000","last_fullsized":"2024-05-08T05:52:33.241408+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.732206+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:51:35.732206+0000","last_peered":"2024-05-08T05:51:35.732206+0000","last_clean":"2024-05-08T05:51:35.732206+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:51:35.732206+0000","last_undegraded":"2024-05-08T05:51:35.732206+0000","last_fullsized":"2024-05-08T05:51:35.732206+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616203+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:51:35.616203+0000","last_peered":"2024-05-08T05:51:35.616203+0000","last_clean":"2024-05-08T05:51:35.616203+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:51:35.616203+0000","last_undegraded":"2024-05-08T05:51:35.616203+0000","last_fullsized":"2024-05-08T05:51:35.616203+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242588+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:52:33.242588+0000","last_peered":"2024-05-08T05:52:33.242588+0000","last_clean":"2024-05-08T05:52:33.242588+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:52:33.242588+0000","last_undegraded":"2024-05-08T05:52:33.242588+0000","last_fullsized":"2024-05-08T05:52:33.242588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.558754+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:51:35.558754+0000","last_peered":"2024-05-08T05:51:35.558754+0000","last_clean":"2024-05-08T05:51:35.558754+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:51:35.558754+0000","last_undegraded":"2024-05-08T05:51:35.558754+0000","last_fullsized":"2024-05-08T05:51:35.558754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.307307+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:51:35.307307+0000","last_peered":"2024-05-08T05:51:35.307307+0000","last_clean":"2024-05-08T05:51:35.307307+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:51:35.307307+0000","last_undegraded":"2024-05-08T05:51:35.307307+0000","last_fullsized":"2024-05-08T05:51:35.307307+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246456+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:52:33.246456+0000","last_peered":"2024-05-08T05:52:33.246456+0000","last_clean":"2024-05-08T05:52:33.246456+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:52:33.246456+0000","last_undegraded":"2024-05-08T05:52:33.246456+0000","last_fullsized":"2024-05-08T05:52:33.246456+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305089+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:51:35.305089+0000","last_peered":"2024-05-08T05:51:35.305089+0000","last_clean":"2024-05-08T05:51:35.305089+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:51:35.305089+0000","last_undegraded":"2024-05-08T05:51:35.305089+0000","last_fullsized":"2024-05-08T05:51:35.305089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.481078+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:51:35.481078+0000","last_peered":"2024-05-08T05:51:35.481078+0000","last_clean":"2024-05-08T05:51:35.481078+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:51:35.481078+0000","last_undegraded":"2024-05-08T05:51:35.481078+0000","last_fullsized":"2024-05-08T05:51:35.481078+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245974+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:52:33.245974+0000","last_peered":"2024-05-08T05:52:33.245974+0000","last_clean":"2024-05-08T05:52:33.245974+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:52:33.245974+0000","last_undegraded":"2024-05-08T05:52:33.245974+0000","last_fullsized":"2024-05-08T05:52:33.245974+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734874+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:51:35.734874+0000","last_peered":"2024-05-08T05:51:35.734874+0000","last_clean":"2024-05-08T05:51:35.734874+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:51:35.734874+0000","last_undegraded":"2024-05-08T05:51:35.734874+0000","last_fullsized":"2024-05-08T05:51:35.734874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616649+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:51:35.616649+0000","last_peered":"2024-05-08T05:51:35.616649+0000","last_clean":"2024-05-08T05:51:35.616649+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:51:35.616649+0000","last_undegraded":"2024-05-08T05:51:35.616649+0000","last_fullsized":"2024-05-08T05:51:35.616649+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245970+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:52:33.245970+0000","last_peered":"2024-05-08T05:52:33.245970+0000","last_clean":"2024-05-08T05:52:33.245970+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:52:33.245970+0000","last_undegraded":"2024-05-08T05:52:33.245970+0000","last_fullsized":"2024-05-08T05:52:33.245970+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306710+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:51:35.306710+0000","last_peered":"2024-05-08T05:51:35.306710+0000","last_clean":"2024-05-08T05:51:35.306710+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:51:35.306710+0000","last_undegraded":"2024-05-08T05:51:35.306710+0000","last_fullsized":"2024-05-08T05:51:35.306710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"89'36","reported_seq":293,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:42.715573+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:52:42.715573+0000","last_peered":"2024-05-08T05:52:42.715573+0000","last_clean":"2024-05-08T05:52:42.715573+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:52:42.715573+0000","last_undegraded":"2024-05-08T05:52:42.715573+0000","last_fullsized":"2024-05-08T05:52:42.715573+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65376,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":212,"num_read_kb":170,"num_write":36,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238558+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:52:33.238558+0000","last_peered":"2024-05-08T05:52:33.238558+0000","last_clean":"2024-05-08T05:52:33.238558+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:52:33.238558+0000","last_undegraded":"2024-05-08T05:52:33.238558+0000","last_fullsized":"2024-05-08T05:52:33.238558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.617056+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:51:35.617056+0000","last_peered":"2024-05-08T05:51:35.617056+0000","last_clean":"2024-05-08T05:51:35.617056+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:51:35.617056+0000","last_undegraded":"2024-05-08T05:51:35.617056+0000","last_fullsized":"2024-05-08T05:51:35.617056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.307011+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:51:35.307011+0000","last_peered":"2024-05-08T05:51:35.307011+0000","last_clean":"2024-05-08T05:51:35.307011+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:51:35.307011+0000","last_undegraded":"2024-05-08T05:51:35.307011+0000","last_fullsized":"2024-05-08T05:51:35.307011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240074+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:52:33.240074+0000","last_peered":"2024-05-08T05:52:33.240074+0000","last_clean":"2024-05-08T05:52:33.240074+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:52:33.240074+0000","last_undegraded":"2024-05-08T05:52:33.240074+0000","last_fullsized":"2024-05-08T05:52:33.240074+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.733865+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:51:35.733865+0000","last_peered":"2024-05-08T05:51:35.733865+0000","last_clean":"2024-05-08T05:51:35.733865+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:51:35.733865+0000","last_undegraded":"2024-05-08T05:51:35.733865+0000","last_fullsized":"2024-05-08T05:51:35.733865+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616588+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:51:35.616588+0000","last_peered":"2024-05-08T05:51:35.616588+0000","last_clean":"2024-05-08T05:51:35.616588+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:51:35.616588+0000","last_undegraded":"2024-05-08T05:51:35.616588+0000","last_fullsized":"2024-05-08T05:51:35.616588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238566+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:52:33.238566+0000","last_peered":"2024-05-08T05:52:33.238566+0000","last_clean":"2024-05-08T05:52:33.238566+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:52:33.238566+0000","last_undegraded":"2024-05-08T05:52:33.238566+0000","last_fullsized":"2024-05-08T05:52:33.238566+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305547+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:51:35.305547+0000","last_peered":"2024-05-08T05:51:35.305547+0000","last_clean":"2024-05-08T05:51:35.305547+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:51:35.305547+0000","last_undegraded":"2024-05-08T05:51:35.305547+0000","last_fullsized":"2024-05-08T05:51:35.305547+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735187+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:51:35.735187+0000","last_peered":"2024-05-08T05:51:35.735187+0000","last_clean":"2024-05-08T05:51:35.735187+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:51:35.735187+0000","last_undegraded":"2024-05-08T05:51:35.735187+0000","last_fullsized":"2024-05-08T05:51:35.735187+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240768+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:52:33.240768+0000","last_peered":"2024-05-08T05:52:33.240768+0000","last_clean":"2024-05-08T05:52:33.240768+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:52:33.240768+0000","last_undegraded":"2024-05-08T05:52:33.240768+0000","last_fullsized":"2024-05-08T05:52:33.240768+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700354+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:51:35.700354+0000","last_peered":"2024-05-08T05:51:35.700354+0000","last_clean":"2024-05-08T05:51:35.700354+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:51:35.700354+0000","last_undegraded":"2024-05-08T05:51:35.700354+0000","last_fullsized":"2024-05-08T05:51:35.700354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":69,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735029+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:51:35.735029+0000","last_peered":"2024-05-08T05:51:35.735029+0000","last_clean":"2024-05-08T05:51:35.735029+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:51:35.735029+0000","last_undegraded":"2024-05-08T05:51:35.735029+0000","last_fullsized":"2024-05-08T05:51:35.735029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246530+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:52:33.246530+0000","last_peered":"2024-05-08T05:52:33.246530+0000","last_clean":"2024-05-08T05:52:33.246530+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:52:33.246530+0000","last_undegraded":"2024-05-08T05:52:33.246530+0000","last_fullsized":"2024-05-08T05:52:33.246530+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616325+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:51:35.616325+0000","last_peered":"2024-05-08T05:51:35.616325+0000","last_clean":"2024-05-08T05:51:35.616325+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:51:35.616325+0000","last_undegraded":"2024-05-08T05:51:35.616325+0000","last_fullsized":"2024-05-08T05:51:35.616325+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735235+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:51:35.735235+0000","last_peered":"2024-05-08T05:51:35.735235+0000","last_clean":"2024-05-08T05:51:35.735235+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:51:35.735235+0000","last_undegraded":"2024-05-08T05:51:35.735235+0000","last_fullsized":"2024-05-08T05:51:35.735235+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.222524+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:52:33.222524+0000","last_peered":"2024-05-08T05:52:33.222524+0000","last_clean":"2024-05-08T05:52:33.222524+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:52:33.222524+0000","last_undegraded":"2024-05-08T05:52:33.222524+0000","last_fullsized":"2024-05-08T05:52:33.222524+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735313+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:51:35.735313+0000","last_peered":"2024-05-08T05:51:35.735313+0000","last_clean":"2024-05-08T05:51:35.735313+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:51:35.735313+0000","last_undegraded":"2024-05-08T05:51:35.735313+0000","last_fullsized":"2024-05-08T05:51:35.735313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305419+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:51:35.305419+0000","last_peered":"2024-05-08T05:51:35.305419+0000","last_clean":"2024-05-08T05:51:35.305419+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:51:35.305419+0000","last_undegraded":"2024-05-08T05:51:35.305419+0000","last_fullsized":"2024-05-08T05:51:35.305419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.241452+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:52:33.241452+0000","last_peered":"2024-05-08T05:52:33.241452+0000","last_clean":"2024-05-08T05:52:33.241452+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:52:33.241452+0000","last_undegraded":"2024-05-08T05:52:33.241452+0000","last_fullsized":"2024-05-08T05:52:33.241452+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700167+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:51:35.700167+0000","last_peered":"2024-05-08T05:51:35.700167+0000","last_clean":"2024-05-08T05:51:35.700167+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:51:35.700167+0000","last_undegraded":"2024-05-08T05:51:35.700167+0000","last_fullsized":"2024-05-08T05:51:35.700167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480772+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:51:35.480772+0000","last_peered":"2024-05-08T05:51:35.480772+0000","last_clean":"2024-05-08T05:51:35.480772+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:51:35.480772+0000","last_undegraded":"2024-05-08T05:51:35.480772+0000","last_fullsized":"2024-05-08T05:51:35.480772+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240128+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:52:33.240128+0000","last_peered":"2024-05-08T05:52:33.240128+0000","last_clean":"2024-05-08T05:52:33.240128+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:52:33.240128+0000","last_undegraded":"2024-05-08T05:52:33.240128+0000","last_fullsized":"2024-05-08T05:52:33.240128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700269+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:51:35.700269+0000","last_peered":"2024-05-08T05:51:35.700269+0000","last_clean":"2024-05-08T05:51:35.700269+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:51:35.700269+0000","last_undegraded":"2024-05-08T05:51:35.700269+0000","last_fullsized":"2024-05-08T05:51:35.700269+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.237143+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:52:33.237143+0000","last_peered":"2024-05-08T05:52:33.237143+0000","last_clean":"2024-05-08T05:52:33.237143+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:52:33.237143+0000","last_undegraded":"2024-05-08T05:52:33.237143+0000","last_fullsized":"2024-05-08T05:52:33.237143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.699860+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:51:35.699860+0000","last_peered":"2024-05-08T05:51:35.699860+0000","last_clean":"2024-05-08T05:51:35.699860+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:51:35.699860+0000","last_undegraded":"2024-05-08T05:51:35.699860+0000","last_fullsized":"2024-05-08T05:51:35.699860+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480986+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:51:35.480986+0000","last_peered":"2024-05-08T05:51:35.480986+0000","last_clean":"2024-05-08T05:51:35.480986+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:51:35.480986+0000","last_undegraded":"2024-05-08T05:51:35.480986+0000","last_fullsized":"2024-05-08T05:51:35.480986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.247419+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:52:33.247419+0000","last_peered":"2024-05-08T05:52:33.247419+0000","last_clean":"2024-05-08T05:52:33.247419+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:52:33.247419+0000","last_undegraded":"2024-05-08T05:52:33.247419+0000","last_fullsized":"2024-05-08T05:52:33.247419+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734647+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:51:35.734647+0000","last_peered":"2024-05-08T05:51:35.734647+0000","last_clean":"2024-05-08T05:51:35.734647+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:51:35.734647+0000","last_undegraded":"2024-05-08T05:51:35.734647+0000","last_fullsized":"2024-05-08T05:51:35.734647+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":119,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480363+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:51:35.480363+0000","last_peered":"2024-05-08T05:51:35.480363+0000","last_clean":"2024-05-08T05:51:35.480363+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:51:35.480363+0000","last_undegraded":"2024-05-08T05:51:35.480363+0000","last_fullsized":"2024-05-08T05:51:35.480363+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616764+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:51:35.616764+0000","last_peered":"2024-05-08T05:51:35.616764+0000","last_clean":"2024-05-08T05:51:35.616764+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:51:35.616764+0000","last_undegraded":"2024-05-08T05:51:35.616764+0000","last_fullsized":"2024-05-08T05:51:35.616764+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.222563+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:52:33.222563+0000","last_peered":"2024-05-08T05:52:33.222563+0000","last_clean":"2024-05-08T05:52:33.222563+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:52:33.222563+0000","last_undegraded":"2024-05-08T05:52:33.222563+0000","last_fullsized":"2024-05-08T05:52:33.222563+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.480616+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:51:35.480616+0000","last_peered":"2024-05-08T05:51:35.480616+0000","last_clean":"2024-05-08T05:51:35.480616+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:51:35.480616+0000","last_undegraded":"2024-05-08T05:51:35.480616+0000","last_fullsized":"2024-05-08T05:51:35.480616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734737+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:51:35.734737+0000","last_peered":"2024-05-08T05:51:35.734737+0000","last_clean":"2024-05-08T05:51:35.734737+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:51:35.734737+0000","last_undegraded":"2024-05-08T05:51:35.734737+0000","last_fullsized":"2024-05-08T05:51:35.734737+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735565+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:51:35.735565+0000","last_peered":"2024-05-08T05:51:35.735565+0000","last_clean":"2024-05-08T05:51:35.735565+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:51:35.735565+0000","last_undegraded":"2024-05-08T05:51:35.735565+0000","last_fullsized":"2024-05-08T05:51:35.735565+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.231031+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:52:33.231031+0000","last_peered":"2024-05-08T05:52:33.231031+0000","last_clean":"2024-05-08T05:52:33.231031+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:52:33.231031+0000","last_undegraded":"2024-05-08T05:52:33.231031+0000","last_fullsized":"2024-05-08T05:52:33.231031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.558480+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:51:35.558480+0000","last_peered":"2024-05-08T05:51:35.558480+0000","last_clean":"2024-05-08T05:51:35.558480+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:51:35.558480+0000","last_undegraded":"2024-05-08T05:51:35.558480+0000","last_fullsized":"2024-05-08T05:51:35.558480+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'11","reported_seq":56,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:06.096275+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:52:06.096275+0000","last_peered":"2024-05-08T05:52:06.096275+0000","last_clean":"2024-05-08T05:52:06.096275+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:52:06.096275+0000","last_undegraded":"2024-05-08T05:52:06.096275+0000","last_fullsized":"2024-05-08T05:52:06.096275+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240434+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:52:33.240434+0000","last_peered":"2024-05-08T05:52:33.240434+0000","last_clean":"2024-05-08T05:52:33.240434+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:52:33.240434+0000","last_undegraded":"2024-05-08T05:52:33.240434+0000","last_fullsized":"2024-05-08T05:52:33.240434+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306587+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:51:35.306587+0000","last_peered":"2024-05-08T05:51:35.306587+0000","last_clean":"2024-05-08T05:51:35.306587+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:51:35.306587+0000","last_undegraded":"2024-05-08T05:51:35.306587+0000","last_fullsized":"2024-05-08T05:51:35.306587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735111+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:51:35.735111+0000","last_peered":"2024-05-08T05:51:35.735111+0000","last_clean":"2024-05-08T05:51:35.735111+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:51:35.735111+0000","last_undegraded":"2024-05-08T05:51:35.735111+0000","last_fullsized":"2024-05-08T05:51:35.735111+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.236016+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:52:33.236016+0000","last_peered":"2024-05-08T05:52:33.236016+0000","last_clean":"2024-05-08T05:52:33.236016+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:52:33.236016+0000","last_undegraded":"2024-05-08T05:52:33.236016+0000","last_fullsized":"2024-05-08T05:52:33.236016+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734828+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:51:35.734828+0000","last_peered":"2024-05-08T05:51:35.734828+0000","last_clean":"2024-05-08T05:51:35.734828+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:51:35.734828+0000","last_undegraded":"2024-05-08T05:51:35.734828+0000","last_fullsized":"2024-05-08T05:51:35.734828+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734814+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:51:35.734814+0000","last_peered":"2024-05-08T05:51:35.734814+0000","last_clean":"2024-05-08T05:51:35.734814+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:51:35.734814+0000","last_undegraded":"2024-05-08T05:51:35.734814+0000","last_fullsized":"2024-05-08T05:51:35.734814+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.220317+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:52:33.220317+0000","last_peered":"2024-05-08T05:52:33.220317+0000","last_clean":"2024-05-08T05:52:33.220317+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:52:33.220317+0000","last_undegraded":"2024-05-08T05:52:33.220317+0000","last_fullsized":"2024-05-08T05:52:33.220317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.305660+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:51:35.305660+0000","last_peered":"2024-05-08T05:51:35.305660+0000","last_clean":"2024-05-08T05:51:35.305660+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:51:35.305660+0000","last_undegraded":"2024-05-08T05:51:35.305660+0000","last_fullsized":"2024-05-08T05:51:35.305660+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.558899+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:51:35.558899+0000","last_peered":"2024-05-08T05:51:35.558899+0000","last_clean":"2024-05-08T05:51:35.558899+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:51:35.558899+0000","last_undegraded":"2024-05-08T05:51:35.558899+0000","last_fullsized":"2024-05-08T05:51:35.558899+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.237013+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:52:33.237013+0000","last_peered":"2024-05-08T05:52:33.237013+0000","last_clean":"2024-05-08T05:52:33.237013+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:52:33.237013+0000","last_undegraded":"2024-05-08T05:52:33.237013+0000","last_fullsized":"2024-05-08T05:52:33.237013+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734948+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:51:35.734948+0000","last_peered":"2024-05-08T05:51:35.734948+0000","last_clean":"2024-05-08T05:51:35.734948+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:51:35.734948+0000","last_undegraded":"2024-05-08T05:51:35.734948+0000","last_fullsized":"2024-05-08T05:51:35.734948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306747+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:51:35.306747+0000","last_peered":"2024-05-08T05:51:35.306747+0000","last_clean":"2024-05-08T05:51:35.306747+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:51:35.306747+0000","last_undegraded":"2024-05-08T05:51:35.306747+0000","last_fullsized":"2024-05-08T05:51:35.306747+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.236615+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:52:33.236615+0000","last_peered":"2024-05-08T05:52:33.236615+0000","last_clean":"2024-05-08T05:52:33.236615+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:52:33.236615+0000","last_undegraded":"2024-05-08T05:52:33.236615+0000","last_fullsized":"2024-05-08T05:52:33.236615+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.734602+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:51:35.734602+0000","last_peered":"2024-05-08T05:51:35.734602+0000","last_clean":"2024-05-08T05:51:35.734602+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:51:35.734602+0000","last_undegraded":"2024-05-08T05:51:35.734602+0000","last_fullsized":"2024-05-08T05:51:35.734602+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735276+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:51:35.735276+0000","last_peered":"2024-05-08T05:51:35.735276+0000","last_clean":"2024-05-08T05:51:35.735276+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:51:35.735276+0000","last_undegraded":"2024-05-08T05:51:35.735276+0000","last_fullsized":"2024-05-08T05:51:35.735276+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244107+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:52:33.244107+0000","last_peered":"2024-05-08T05:52:33.244107+0000","last_clean":"2024-05-08T05:52:33.244107+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:52:33.244107+0000","last_undegraded":"2024-05-08T05:52:33.244107+0000","last_fullsized":"2024-05-08T05:52:33.244107+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.735740+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:51:35.735740+0000","last_peered":"2024-05-08T05:51:35.735740+0000","last_clean":"2024-05-08T05:51:35.735740+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:51:35.735740+0000","last_undegraded":"2024-05-08T05:51:35.735740+0000","last_fullsized":"2024-05-08T05:51:35.735740+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.306985+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:51:35.306985+0000","last_peered":"2024-05-08T05:51:35.306985+0000","last_clean":"2024-05-08T05:51:35.306985+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:51:35.306985+0000","last_undegraded":"2024-05-08T05:51:35.306985+0000","last_fullsized":"2024-05-08T05:51:35.306985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244500+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:52:33.244500+0000","last_peered":"2024-05-08T05:52:33.244500+0000","last_clean":"2024-05-08T05:52:33.244500+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:52:33.244500+0000","last_undegraded":"2024-05-08T05:52:33.244500+0000","last_fullsized":"2024-05-08T05:52:33.244500+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.731449+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:51:35.731449+0000","last_peered":"2024-05-08T05:51:35.731449+0000","last_clean":"2024-05-08T05:51:35.731449+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:51:35.731449+0000","last_undegraded":"2024-05-08T05:51:35.731449+0000","last_fullsized":"2024-05-08T05:51:35.731449+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"78'1","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616933+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:51:35.616933+0000","last_peered":"2024-05-08T05:51:35.616933+0000","last_clean":"2024-05-08T05:51:35.616933+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:51:35.616933+0000","last_undegraded":"2024-05-08T05:51:35.616933+0000","last_fullsized":"2024-05-08T05:51:35.616933+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.240971+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:52:33.240971+0000","last_peered":"2024-05-08T05:52:33.240971+0000","last_clean":"2024-05-08T05:52:33.240971+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:52:33.240971+0000","last_undegraded":"2024-05-08T05:52:33.240971+0000","last_fullsized":"2024-05-08T05:52:33.240971+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.616156+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:51:35.616156+0000","last_peered":"2024-05-08T05:51:35.616156+0000","last_clean":"2024-05-08T05:51:35.616156+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:51:35.616156+0000","last_undegraded":"2024-05-08T05:51:35.616156+0000","last_fullsized":"2024-05-08T05:51:35.616156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:51:35.700459+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:51:35.700459+0000","last_peered":"2024-05-08T05:51:35.700459+0000","last_clean":"2024-05-08T05:51:35.700459+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:51:35.700459+0000","last_undegraded":"2024-05-08T05:51:35.700459+0000","last_fullsized":"2024-05-08T05:51:35.700459+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238584+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:52:33.238584+0000","last_peered":"2024-05-08T05:52:33.238584+0000","last_clean":"2024-05-08T05:52:33.238584+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:52:33.238584+0000","last_undegraded":"2024-05-08T05:52:33.238584+0000","last_fullsized":"2024-05-08T05:52:33.238584+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.238497+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:52:33.238497+0000","last_peered":"2024-05-08T05:52:33.238497+0000","last_clean":"2024-05-08T05:52:33.238497+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:52:33.238497+0000","last_undegraded":"2024-05-08T05:52:33.238497+0000","last_fullsized":"2024-05-08T05:52:33.238497+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":51,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.241912+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:52:33.241912+0000","last_peered":"2024-05-08T05:52:33.241912+0000","last_clean":"2024-05-08T05:52:33.241912+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:52:33.241912+0000","last_undegraded":"2024-05-08T05:52:33.241912+0000","last_fullsized":"2024-05-08T05:52:33.241912+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245814+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:52:33.245814+0000","last_peered":"2024-05-08T05:52:33.245814+0000","last_clean":"2024-05-08T05:52:33.245814+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:52:33.245814+0000","last_undegraded":"2024-05-08T05:52:33.245814+0000","last_fullsized":"2024-05-08T05:52:33.245814+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.245127+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:52:33.245127+0000","last_peered":"2024-05-08T05:52:33.245127+0000","last_clean":"2024-05-08T05:52:33.245127+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:52:33.245127+0000","last_undegraded":"2024-05-08T05:52:33.245127+0000","last_fullsized":"2024-05-08T05:52:33.245127+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.244876+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:52:33.244876+0000","last_peered":"2024-05-08T05:52:33.244876+0000","last_clean":"2024-05-08T05:52:33.244876+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:52:33.244876+0000","last_undegraded":"2024-05-08T05:52:33.244876+0000","last_fullsized":"2024-05-08T05:52:33.244876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.223223+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:52:33.223223+0000","last_peered":"2024-05-08T05:52:33.223223+0000","last_clean":"2024-05-08T05:52:33.223223+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:52:33.223223+0000","last_undegraded":"2024-05-08T05:52:33.223223+0000","last_fullsized":"2024-05-08T05:52:33.223223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246828+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:52:33.246828+0000","last_peered":"2024-05-08T05:52:33.246828+0000","last_clean":"2024-05-08T05:52:33.246828+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:52:33.246828+0000","last_undegraded":"2024-05-08T05:52:33.246828+0000","last_fullsized":"2024-05-08T05:52:33.246828+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.236084+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:52:33.236084+0000","last_peered":"2024-05-08T05:52:33.236084+0000","last_clean":"2024-05-08T05:52:33.236084+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:52:33.236084+0000","last_undegraded":"2024-05-08T05:52:33.236084+0000","last_fullsized":"2024-05-08T05:52:33.236084+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.242312+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:52:33.242312+0000","last_peered":"2024-05-08T05:52:33.242312+0000","last_clean":"2024-05-08T05:52:33.242312+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:52:33.242312+0000","last_undegraded":"2024-05-08T05:52:33.242312+0000","last_fullsized":"2024-05-08T05:52:33.242312+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.222572+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:52:33.222572+0000","last_peered":"2024-05-08T05:52:33.222572+0000","last_clean":"2024-05-08T05:52:33.222572+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:52:33.222572+0000","last_undegraded":"2024-05-08T05:52:33.222572+0000","last_fullsized":"2024-05-08T05:52:33.222572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246246+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:52:33.246246+0000","last_peered":"2024-05-08T05:52:33.246246+0000","last_clean":"2024-05-08T05:52:33.246246+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:52:33.246246+0000","last_undegraded":"2024-05-08T05:52:33.246246+0000","last_fullsized":"2024-05-08T05:52:33.246246+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":49,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.246412+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:52:33.246412+0000","last_peered":"2024-05-08T05:52:33.246412+0000","last_clean":"2024-05-08T05:52:33.246412+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:52:33.246412+0000","last_undegraded":"2024-05-08T05:52:33.246412+0000","last_fullsized":"2024-05-08T05:52:33.246412+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-05-08T05:52:33.234119+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:52:33.234119+0000","last_peered":"2024-05-08T05:52:33.234119+0000","last_clean":"2024-05-08T05:52:33.234119+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:52:33.234119+0000","last_undegraded":"2024-05-08T05:52:33.234119+0000","last_fullsized":"2024-05-08T05:52:33.234119+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280284,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66174,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":218,"num_read_kb":176,"num_write":95,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":198522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710769,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":514408,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906998,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":1166250,"data_compressed":68051,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070524,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":519086,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266754,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":515012,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462982,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":1109454,"data_compressed":68051,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626508,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":510312,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574947,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":575688,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":510402,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":575722,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":978944,"data_stored":510416,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986262,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":514430,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822736,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1104958,"data_compressed":68051,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34023,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":65410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:52:47.779 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:52:47.779 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T05:52:47.779 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T05:52:47.779 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T05:52:48.033 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:48.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:48 smithi003 ceph-mon[30137]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:48.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:48 smithi003 ceph-mon[30137]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-05-08T05:52:49.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:48 smithi089 ceph-mon[34495]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:49.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:48 smithi089 ceph-mon[34495]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-05-08T05:52:49.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:48 smithi104 ceph-mon[34559]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:52:49.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:48 smithi104 ceph-mon[34559]: pgmap v386: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 0 B/s wr, 1 op/s 2024-05-08T05:52:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.851+0000 7f9b04ba6700 1 -- 172.21.15.3:0/4021641760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00102890 msgr2=0x7f9b00110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.851+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/4021641760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00102890 0x7f9b00110390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9af800b0a0 tx=0x7f9af800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.852 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.851+0000 7f9b04ba6700 1 -- 172.21.15.3:0/4021641760 shutdown_connections 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.851+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/4021641760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00102890 0x7f9b00110390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/4021641760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b00109680 0x7f9b00109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/4021641760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00101ed0 0x7f9b00102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 -- 172.21.15.3:0/4021641760 >> 172.21.15.3:0/4021641760 conn(0x7f9b00077610 msgr2=0x7f9b00077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 -- 172.21.15.3:0/4021641760 shutdown_connections 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 -- 172.21.15.3:0/4021641760 wait complete. 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 Processor -- start 2024-05-08T05:52:49.853 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.852+0000 7f9b04ba6700 1 -- start start 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9b04ba6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 0x7f9b0019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9b04ba6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b00102890 0x7f9b0019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9b04ba6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 0x7f9b00199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9b04ba6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001136c0 con 0x7f9b00101ed0 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9b04ba6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00113540 con 0x7f9b00109680 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9b04ba6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00113840 con 0x7f9b00102890 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 0x7f9b0019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 0x7f9b00199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 0x7f9b0019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:51952/0 (socket says 172.21.15.3:51952) 2024-05-08T05:52:49.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 0x7f9b00199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:50650/0 (socket says 172.21.15.3:50650) 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 -- 172.21.15.3:0/1659914490 learned_addr learned my addr 172.21.15.3:0/1659914490 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afdd9b700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b00102890 0x7f9b0019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 -- 172.21.15.3:0/1659914490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b00102890 msgr2=0x7f9b0019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b00102890 0x7f9b0019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 -- 172.21.15.3:0/1659914490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 msgr2=0x7f9b00199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 0x7f9b00199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 -- 172.21.15.3:0/1659914490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0019a080 con 0x7f9b00101ed0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afed9d700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 0x7f9b00199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.853+0000 7f9afe59c700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 0x7f9b0019f760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b530 tx=0x7f9af000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.854+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0015020 con 0x7f9b00101ed0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.854+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0019a370 con 0x7f9b00101ed0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.854+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9af0011ce0 con 0x7f9b00101ed0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.854+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af0010dd0 con 0x7f9b00101ed0 2024-05-08T05:52:49.856 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.854+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b001a6630 con 0x7f9b00101ed0 2024-05-08T05:52:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.855+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b0010bac0 con 0x7f9b00101ed0 2024-05-08T05:52:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.857+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9af0018020 con 0x7f9b00101ed0 2024-05-08T05:52:49.857 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.858+0000 7f9aef7fe700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9ae80b2900 0x7f9ae80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.858+0000 7f9afdd9b700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9ae80b2900 0x7f9ae80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.858+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9af00e21d0 con 0x7f9b00101ed0 2024-05-08T05:52:49.858 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.858+0000 7f9afdd9b700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9ae80b2900 0x7f9ae80b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b00077b30 tx=0x7f9af400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:49.861 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:49.861+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9af001f020 con 0x7f9b00101ed0 2024-05-08T05:52:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.123+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9b0006a190 con 0x7f9b00101ed0 2024-05-08T05:52:50.124 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.124+0000 7f9aef7fe700 1 -- 172.21.15.3:0/1659914490 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9af00a9b60 con 0x7f9b00101ed0 2024-05-08T05:52:50.124 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:52:50.124 INFO:teuthology.orchestra.run.smithi003.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:52:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9ae80b2900 msgr2=0x7f9ae80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9ae80b2900 0x7f9ae80b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b00077b30 tx=0x7f9af400b410 comp rx=0 tx=0).stop 2024-05-08T05:52:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 msgr2=0x7f9b0019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 0x7f9b0019f760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b530 tx=0x7f9af000b840 comp rx=0 tx=0).stop 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 shutdown_connections 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b00109680 0x7f9b00199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9ae80b2900 0x7f9ae80b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b00101ed0 0x7f9b0019f760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 --2- 172.21.15.3:0/1659914490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b00102890 0x7f9b0019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 >> 172.21.15.3:0/1659914490 conn(0x7f9b00077610 msgr2=0x7f9b00100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 shutdown_connections 2024-05-08T05:52:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:50.126+0000 7f9b04ba6700 1 -- 172.21.15.3:0/1659914490 wait complete. 2024-05-08T05:52:50.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:50 smithi003 ceph-mon[30137]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-08T05:52:50.843 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1659914490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:52:50.873 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T05:52:50.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:52:51.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:50 smithi089 ceph-mon[34495]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-08T05:52:51.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1659914490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:52:51.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:50 smithi104 ceph-mon[34559]: pgmap v387: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-08T05:52:51.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1659914490' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:52:51.128 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 -- 172.21.15.3:0/3885851321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4103780 msgr2=0x7fc1a4103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 --2- 172.21.15.3:0/3885851321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4103780 0x7fc1a4103b60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc1940099c0 tx=0x7fc194009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 -- 172.21.15.3:0/3885851321 shutdown_connections 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 --2- 172.21.15.3:0/3885851321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a4108870 0x7fc1a41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 --2- 172.21.15.3:0/3885851321 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4103780 0x7fc1a4103b60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 --2- 172.21.15.3:0/3885851321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a4104130 0x7fc1a4108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 -- 172.21.15.3:0/3885851321 >> 172.21.15.3:0/3885851321 conn(0x7fc1a407a610 msgr2=0x7fc1a407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 -- 172.21.15.3:0/3885851321 shutdown_connections 2024-05-08T05:52:52.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.927+0000 7fc1aa112700 1 -- 172.21.15.3:0/3885851321 wait complete. 2024-05-08T05:52:52.929 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:52 smithi003 ceph-mon[30137]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.928+0000 7fc1aa112700 1 Processor -- start 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.928+0000 7fc1aa112700 1 -- start start 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1aa112700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a4103780 0x7fc1a419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1aa112700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a4104130 0x7fc1a419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1aa112700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 0x7fc1a41999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1aa112700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4113730 con 0x7fc1a4108870 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1aa112700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41135b0 con 0x7fc1a4104130 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1aa112700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41138b0 con 0x7fc1a4103780 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 0x7fc1a41999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 0x7fc1a41999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39294/0 (socket says 172.21.15.3:39294) 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 -- 172.21.15.3:0/2302361575 learned_addr learned my addr 172.21.15.3:0/2302361575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a37fe700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a4103780 0x7fc1a419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:52.931 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc19affd700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a4104130 0x7fc1a419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 -- 172.21.15.3:0/2302361575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a4103780 msgr2=0x7fc1a419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a4103780 0x7fc1a419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 -- 172.21.15.3:0/2302361575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a4104130 msgr2=0x7fc1a419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a4104130 0x7fc1a419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 -- 172.21.15.3:0/2302361575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a419a150 con 0x7fc1a4108870 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.929+0000 7fc1a3fff700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 0x7fc1a41999f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b970 tx=0x7fc19c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.930+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c019070 con 0x7fc1a4108870 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.930+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c00cde0 con 0x7fc1a4108870 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.930+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a419a440 con 0x7fc1a4108870 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.930+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c007370 con 0x7fc1a4108870 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.930+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a40756d0 con 0x7fc1a4108870 2024-05-08T05:52:52.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.931+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a410bab0 con 0x7fc1a4108870 2024-05-08T05:52:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.932+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fc19c01c070 con 0x7fc1a4108870 2024-05-08T05:52:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.933+0000 7fc1a17fa700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc1900b2880 0x7fc1900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.933+0000 7fc1a37fe700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc1900b2880 0x7fc1900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:52.933 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.933+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fc19c0e2200 con 0x7fc1a4108870 2024-05-08T05:52:52.934 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.934+0000 7fc1a37fe700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc1900b2880 0x7fc1900b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1a419ab30 tx=0x7fc19400b870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:52.937 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:52.937+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc19c023070 con 0x7fc1a4108870 2024-05-08T05:52:53.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:52 smithi089 ceph-mon[34495]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:53.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:52 smithi104 ceph-mon[34559]: pgmap v388: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.198+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1a419af20 con 0x7fc1a4108870 2024-05-08T05:52:53.199 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.199+0000 7fc1a17fa700 1 -- 172.21.15.3:0/2302361575 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7fc19c0a9b90 con 0x7fc1a4108870 2024-05-08T05:52:53.200 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:52:53.200 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.201+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc1900b2880 msgr2=0x7fc1900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.201+0000 7fc1aa112700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc1900b2880 0x7fc1900b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc1a419ab30 tx=0x7fc19400b870 comp rx=0 tx=0).stop 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.201+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 msgr2=0x7fc1a41999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.201+0000 7fc1aa112700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 0x7fc1a41999f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00b970 tx=0x7fc19c00bc80 comp rx=0 tx=0).stop 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 shutdown_connections 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fc1900b2880 0x7fc1900b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fc1a4108870 0x7fc1a41999f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a4103780 0x7fc1a419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:53.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 --2- 172.21.15.3:0/2302361575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc1a4104130 0x7fc1a419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 >> 172.21.15.3:0/2302361575 conn(0x7fc1a407a610 msgr2=0x7fc1a41017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 shutdown_connections 2024-05-08T05:52:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:53.202+0000 7fc1aa112700 1 -- 172.21.15.3:0/2302361575 wait complete. 2024-05-08T05:52:53.203 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:52:53.798 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:52:53.823 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:53 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2302361575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:52:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:53 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2302361575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:52:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:53 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2302361575' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:52:54.050 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:54.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:54 smithi003 ceph-mon[30137]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:55.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:54 smithi089 ceph-mon[34495]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:55.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:54 smithi104 ceph-mon[34559]: pgmap v389: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 -- 172.21.15.3:0/587061749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 msgr2=0x7f531c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 --2- 172.21.15.3:0/587061749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c10f250 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f531400b0a0 tx=0x7f531400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 -- 172.21.15.3:0/587061749 shutdown_connections 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 --2- 172.21.15.3:0/587061749 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c10f250 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 --2- 172.21.15.3:0/587061749 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 0x7f531c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 --2- 172.21.15.3:0/587061749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:55.953 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.950+0000 7f5324617700 1 -- 172.21.15.3:0/587061749 >> 172.21.15.3:0/587061749 conn(0x7f531c079350 msgr2=0x7f531c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 -- 172.21.15.3:0/587061749 shutdown_connections 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 -- 172.21.15.3:0/587061749 wait complete. 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 Processor -- start 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 -- start start 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.951+0000 7f5324617700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 0x7f531c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5324617700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f531c1125b0 con 0x7f531c102d60 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5324617700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112430 con 0x7f531c108410 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5324617700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f531c112730 con 0x7f531c1023a0 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:55.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f53223b3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f53223b3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:55398/0 (socket says 172.21.15.3:55398) 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39314/0 (socket says 172.21.15.3:39314) 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f53223b3700 1 -- 172.21.15.3:0/956642782 learned_addr learned my addr 172.21.15.3:0/956642782 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5322bb4700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 0x7f531c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 -- 172.21.15.3:0/956642782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 msgr2=0x7f531c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 -- 172.21.15.3:0/956642782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 msgr2=0x7f531c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 0x7f531c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 -- 172.21.15.3:0/956642782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c1a8090 con 0x7f531c102d60 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f53223b3700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5322bb4700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 0x7f531c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f5321bb2700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c1045c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b450 tx=0x7f530c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c015020 con 0x7f531c102d60 2024-05-08T05:52:55.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.952+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f530c011ce0 con 0x7f531c102d60 2024-05-08T05:52:55.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.953+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c1a8380 con 0x7f531c102d60 2024-05-08T05:52:55.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.953+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c010cf0 con 0x7f531c102d60 2024-05-08T05:52:55.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.953+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c1a88c0 con 0x7f531c102d60 2024-05-08T05:52:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.953+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c06a190 con 0x7f531c102d60 2024-05-08T05:52:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.955+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f530c018020 con 0x7f531c102d60 2024-05-08T05:52:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.956+0000 7f53137fe700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53080b2880 0x7f53080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.956+0000 7f53223b3700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53080b2880 0x7f53080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.956+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f530c0e2230 con 0x7f531c102d60 2024-05-08T05:52:55.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.956+0000 7f53223b3700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53080b2880 0x7f53080b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5318005950 tx=0x7f531800a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:55.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:55.959+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f530c01f020 con 0x7f531c102d60 2024-05-08T05:52:56.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.221+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f531c1055b0 con 0x7f531c102d60 2024-05-08T05:52:56.222 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.222+0000 7f53137fe700 1 -- 172.21.15.3:0/956642782 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7f531c1055b0 con 0x7f531c102d60 2024-05-08T05:52:56.222 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:52:56.223 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.224+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53080b2880 msgr2=0x7f53080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.224+0000 7f5324617700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53080b2880 0x7f53080b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5318005950 tx=0x7f531800a400 comp rx=0 tx=0).stop 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.224+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 msgr2=0x7f531c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.224+0000 7f5324617700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c1045c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f530c00b450 tx=0x7f530c00b760 comp rx=0 tx=0).stop 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 shutdown_connections 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f531c108410 0x7f531c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f531c1023a0 0x7f531c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53080b2880 0x7f53080b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 --2- 172.21.15.3:0/956642782 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f531c102d60 0x7f531c1045c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 >> 172.21.15.3:0/956642782 conn(0x7f531c079350 msgr2=0x7f531c101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 shutdown_connections 2024-05-08T05:52:56.225 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:56.225+0000 7f5324617700 1 -- 172.21.15.3:0/956642782 wait complete. 2024-05-08T05:52:56.226 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:52:56.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:56 smithi003 ceph-mon[30137]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:56.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:56 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/956642782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:52:56.978 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:52:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:56 smithi089 ceph-mon[34495]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:56 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/956642782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:52:57.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:56 smithi104 ceph-mon[34559]: pgmap v390: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:52:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:56 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/956642782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:52:57.274 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:52:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:58 smithi089 ceph-mon[34495]: pgmap v391: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:59.017 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:58 smithi003 ceph-mon[30137]: pgmap v391: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:59.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:58 smithi104 ceph-mon[34559]: pgmap v391: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:52:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.124+0000 7f42685a3700 1 -- 172.21.15.3:0/555345300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260104130 msgr2=0x7f4260108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.124+0000 7f42685a3700 1 --2- 172.21.15.3:0/555345300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260104130 0x7f4260108140 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f42500099c0 tx=0x7f4250009cd0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 -- 172.21.15.3:0/555345300 shutdown_connections 2024-05-08T05:52:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 --2- 172.21.15.3:0/555345300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260108870 0x7f42601103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.125 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 --2- 172.21.15.3:0/555345300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 0x7f4260103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 --2- 172.21.15.3:0/555345300 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260104130 0x7f4260108140 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 -- 172.21.15.3:0/555345300 >> 172.21.15.3:0/555345300 conn(0x7f426007a610 msgr2=0x7f426007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 -- 172.21.15.3:0/555345300 shutdown_connections 2024-05-08T05:52:59.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 -- 172.21.15.3:0/555345300 wait complete. 2024-05-08T05:52:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.125+0000 7f42685a3700 1 Processor -- start 2024-05-08T05:52:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 -- start start 2024-05-08T05:52:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 0x7f42601a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260104130 0x7f42601a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 0x7f426019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:59.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4260113630 con 0x7f4260108870 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f42601134b0 con 0x7f4260103780 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f42685a3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42601137b0 con 0x7f4260104130 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 0x7f426019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 0x7f426019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39328/0 (socket says 172.21.15.3:39328) 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 -- 172.21.15.3:0/3822400992 learned_addr learned my addr 172.21.15.3:0/3822400992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4265b3e700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260104130 0x7f42601a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f426633f700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 0x7f42601a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 -- 172.21.15.3:0/3822400992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260104130 msgr2=0x7f42601a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260104130 0x7f42601a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 -- 172.21.15.3:0/3822400992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 msgr2=0x7f42601a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 0x7f42601a1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f4266b40700 1 -- 172.21.15.3:0/3822400992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426019c170 con 0x7f4260108870 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.126+0000 7f426633f700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 0x7f42601a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:52:59.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.127+0000 7f4266b40700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 0x7f426019ba10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4258007d70 tx=0x7f425800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.127+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258016020 con 0x7f4260108870 2024-05-08T05:52:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.127+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426019c460 con 0x7f4260108870 2024-05-08T05:52:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.127+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f425800ad40 con 0x7f4260108870 2024-05-08T05:52:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.127+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580115e0 con 0x7f4260108870 2024-05-08T05:52:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.127+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42601a8720 con 0x7f4260108870 2024-05-08T05:52:59.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.128+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426006a190 con 0x7f4260108870 2024-05-08T05:52:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.129+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f4258019020 con 0x7f4260108870 2024-05-08T05:52:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.130+0000 7f42577fe700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f424c0b2880 0x7f424c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:52:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.130+0000 7f426633f700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f424c0b2880 0x7f424c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:52:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.130+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f42580e2920 con 0x7f4260108870 2024-05-08T05:52:59.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.131+0000 7f426633f700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f424c0b2880 0x7f424c0b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f426019cb50 tx=0x7f425c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:52:59.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.133+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4258020020 con 0x7f4260108870 2024-05-08T05:52:59.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.348+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f426019cf40 con 0x7f4260108870 2024-05-08T05:52:59.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.350+0000 7f42577fe700 1 -- 172.21.15.3:0/3822400992 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24666 (secure 0 0 0) 0x7f426019cf40 con 0x7f4260108870 2024-05-08T05:52:59.350 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:52:59.351 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:51:35.298219+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:52:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f424c0b2880 msgr2=0x7f424c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:59.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f424c0b2880 0x7f424c0b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f426019cb50 tx=0x7f425c00a680 comp rx=0 tx=0).stop 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 msgr2=0x7f426019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 0x7f426019ba10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4258007d70 tx=0x7f425800a340 comp rx=0 tx=0).stop 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 shutdown_connections 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f424c0b2880 0x7f424c0b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4260108870 0x7f426019ba10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4260103780 0x7f42601a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 --2- 172.21.15.3:0/3822400992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4260104130 0x7f42601a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 >> 172.21.15.3:0/3822400992 conn(0x7f426007a610 msgr2=0x7f42601016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 shutdown_connections 2024-05-08T05:52:59.353 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:52:59.352+0000 7f42685a3700 1 -- 172.21.15.3:0/3822400992 wait complete. 2024-05-08T05:53:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:52:59 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3822400992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:52:59 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3822400992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:00.139 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:52:59 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3822400992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:00.175 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:00.435 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:00.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:00 smithi003 ceph-mon[30137]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:00 smithi089 ceph-mon[34495]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:00 smithi104 ceph-mon[34559]: pgmap v392: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:02.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 -- 172.21.15.3:0/1899524814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 msgr2=0x7f742c108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 --2- 172.21.15.3:0/1899524814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c108140 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f74200099c0 tx=0x7f7420009cd0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 -- 172.21.15.3:0/1899524814 shutdown_connections 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 --2- 172.21.15.3:0/1899524814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f742c108870 0x7f742c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 --2- 172.21.15.3:0/1899524814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c103780 0x7f742c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 --2- 172.21.15.3:0/1899524814 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c108140 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 -- 172.21.15.3:0/1899524814 >> 172.21.15.3:0/1899524814 conn(0x7f742c07a610 msgr2=0x7f742c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 -- 172.21.15.3:0/1899524814 shutdown_connections 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 -- 172.21.15.3:0/1899524814 wait complete. 2024-05-08T05:53:02.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.284+0000 7f7431379700 1 Processor -- start 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 -- start start 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f742c103780 0x7f742c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c108870 0x7f742c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f742c113630 con 0x7f742c104130 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1134b0 con 0x7f742c108870 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f7431379700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f742c1137b0 con 0x7f742c103780 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c19fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:39334/0 (socket says 172.21.15.3:39334) 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 -- 172.21.15.3:0/3411750059 learned_addr learned my addr 172.21.15.3:0/3411750059 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742affd700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f742c103780 0x7f742c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.288 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742b7fe700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c108870 0x7f742c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 -- 172.21.15.3:0/3411750059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f742c103780 msgr2=0x7f742c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f742c103780 0x7f742c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 -- 172.21.15.3:0/3411750059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c108870 msgr2=0x7f742c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c108870 0x7f742c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 -- 172.21.15.3:0/3411750059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c19a050 con 0x7f742c104130 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.285+0000 7f742a7fc700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c19fbe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f742000b580 tx=0x7f74200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.286+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420027070 con 0x7f742c104130 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.286+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7420004020 con 0x7f742c104130 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.286+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c19a2e0 con 0x7f742c104130 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.286+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420022e10 con 0x7f742c104130 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.286+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1a6500 con 0x7f742c104130 2024-05-08T05:53:02.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.287+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c104c00 con 0x7f742c104130 2024-05-08T05:53:02.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.288+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f742002a070 con 0x7f742c104130 2024-05-08T05:53:02.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.289+0000 7f7413fff700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74140b2880 0x7f74140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:02.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.289+0000 7f742affd700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74140b2880 0x7f74140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:02.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.289+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f74200e3440 con 0x7f742c104130 2024-05-08T05:53:02.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.290+0000 7f742affd700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74140b2880 0x7f74140b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f741c00ac50 tx=0x7f741c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:02.293 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.293+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7420013070 con 0x7f742c104130 2024-05-08T05:53:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.557+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f742c19ad50 con 0x7f742c104130 2024-05-08T05:53:02.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.558+0000 7f7413fff700 1 -- 172.21.15.3:0/3411750059 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7f74200aae60 con 0x7f742c104130 2024-05-08T05:53:02.558 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:02.559 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74140b2880 msgr2=0x7f74140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74140b2880 0x7f74140b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f741c00ac50 tx=0x7f741c00a380 comp rx=0 tx=0).stop 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 msgr2=0x7f742c19fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c19fbe0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f742000b580 tx=0x7f74200047b0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 shutdown_connections 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f742c108870 0x7f742c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f742c103780 0x7f742c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f74140b2880 0x7f74140b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 --2- 172.21.15.3:0/3411750059 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f742c104130 0x7f742c19fbe0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:02.561 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 >> 172.21.15.3:0/3411750059 conn(0x7f742c07a610 msgr2=0x7f742c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:02.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 shutdown_connections 2024-05-08T05:53:02.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:02.561+0000 7f7431379700 1 -- 172.21.15.3:0/3411750059 wait complete. 2024-05-08T05:53:02.562 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:53:02.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:02 smithi003 ceph-mon[30137]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:02.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:02 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3411750059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:02 smithi089 ceph-mon[34495]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:03.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:02 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3411750059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:03.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:02 smithi104 ceph-mon[34559]: pgmap v393: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:03.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:02 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3411750059' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:03.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:50:01.809365+0000', 'modified': '2024-05-08T05:51:05.493748+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6839/472688413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 472688413}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 472688413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.104:6839/3320987620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3320987620}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3320987620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24493]} 2024-05-08T05:53:03.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:53:03.363 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:53:03.619 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:04.811 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:04 smithi003 ceph-mon[30137]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:05.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:04 smithi089 ceph-mon[34495]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:05.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:04 smithi104 ceph-mon[34559]: pgmap v394: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 -- 172.21.15.3:0/1911424027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 msgr2=0x7f998010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 --2- 172.21.15.3:0/1911424027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998010f250 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f997800b0a0 tx=0x7f997800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 -- 172.21.15.3:0/1911424027 shutdown_connections 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 --2- 172.21.15.3:0/1911424027 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998010f250 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 --2- 172.21.15.3:0/1911424027 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9980108410 0x7f99801087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 --2- 172.21.15.3:0/1911424027 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f99801023a0 0x7f9980102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 -- 172.21.15.3:0/1911424027 >> 172.21.15.3:0/1911424027 conn(0x7f9980079350 msgr2=0x7f9980079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.315+0000 7f99872ac700 1 -- 172.21.15.3:0/1911424027 shutdown_connections 2024-05-08T05:53:05.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.316+0000 7f99872ac700 1 -- 172.21.15.3:0/1911424027 wait complete. 2024-05-08T05:53:05.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.316+0000 7f99872ac700 1 Processor -- start 2024-05-08T05:53:05.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.316+0000 7f99872ac700 1 -- start start 2024-05-08T05:53:05.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f99872ac700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99801023a0 0x7f998019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f99872ac700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998019b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f99872ac700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9980108410 0x7f99801a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f99872ac700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9980112630 con 0x7f9980102d60 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f99872ac700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f99801124b0 con 0x7f9980108410 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f99872ac700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99801127b0 con 0x7f99801023a0 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998019b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998019b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56160/0 (socket says 172.21.15.3:56160) 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 -- 172.21.15.3:0/1038148649 learned_addr learned my addr 172.21.15.3:0/1038148649 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9985048700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99801023a0 0x7f998019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9985849700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9980108410 0x7f99801a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 -- 172.21.15.3:0/1038148649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99801023a0 msgr2=0x7f998019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99801023a0 0x7f998019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 -- 172.21.15.3:0/1038148649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9980108410 msgr2=0x7f99801a5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9980108410 0x7f99801a5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 -- 172.21.15.3:0/1038148649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99801a6090 con 0x7f9980102d60 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.317+0000 7f9984847700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998019b010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f997000b530 tx=0x7f997000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.318+0000 7f99767fc700 1 -- 172.21.15.3:0/1038148649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970015020 con 0x7f9980102d60 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.318+0000 7f99767fc700 1 -- 172.21.15.3:0/1038148649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9970011ce0 con 0x7f9980102d60 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.318+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99801a62c0 con 0x7f9980102d60 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.318+0000 7f99767fc700 1 -- 172.21.15.3:0/1038148649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9970010dd0 con 0x7f9980102d60 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.318+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99801a6800 con 0x7f9980102d60 2024-05-08T05:53:05.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.320+0000 7f99767fc700 1 -- 172.21.15.3:0/1038148649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f9970018020 con 0x7f9980102d60 2024-05-08T05:53:05.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.321+0000 7f99767fc700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f996c0b2880 0x7f996c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.321+0000 7f9985048700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f996c0b2880 0x7f996c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.321+0000 7f99767fc700 1 -- 172.21.15.3:0/1038148649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f99700e1cc0 con 0x7f9980102d60 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.321+0000 7f9991b3fb80 1 Processor -- start 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.321+0000 7f9991b3fb80 1 -- start start 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.321+0000 7f9985048700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f996c0b2880 0x7f996c0b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f997c009a10 tx=0x7f997c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe5890 0x55601cbe5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbe9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe9ff0 0x55601cbea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55601cb33810 con 0x55601cbe61b0 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55601cad2770 con 0x55601cbe9ff0 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55601cb2f7d0 con 0x55601cbe5890 2024-05-08T05:53:05.323 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbe9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbe9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56170/0 (socket says 172.21.15.3:56170) 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 -- 172.21.15.3:0/131884040 learned_addr learned my addr 172.21.15.3:0/131884040 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9985048700 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe5890 0x55601cbe5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9985849700 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe9ff0 0x55601cbea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 -- 172.21.15.3:0/131884040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe5890 msgr2=0x55601cbe5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe5890 0x55601cbe5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 -- 172.21.15.3:0/131884040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe9ff0 msgr2=0x55601cbea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe9ff0 0x55601cbea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 -- 172.21.15.3:0/131884040 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601ca80880 con 0x55601cbe61b0 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9985849700 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe9ff0 0x55601cbea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f9984847700 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbe9ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99700a6160 tx=0x7f99700a8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea4863fcc42520e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.322+0000 7f996b7fe700 1 -- 172.21.15.3:0/131884040 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99700a95f0 con 0x55601cbe61b0 2024-05-08T05:53:05.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f996b7fe700 1 -- 172.21.15.3:0/131884040 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99700a8ef0 con 0x55601cbe61b0 2024-05-08T05:53:05.325 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f996b7fe700 1 -- 172.21.15.3:0/131884040 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99700a8150 con 0x55601cbe61b0 2024-05-08T05:53:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 -- 172.21.15.3:0/131884040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 msgr2=0x55601cbe9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbe9ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99700a6160 tx=0x7f99700a8930 comp rx=0 tx=0).stop 2024-05-08T05:53:05.328 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 -- 172.21.15.3:0/131884040 shutdown_connections 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe9ff0 0x55601cbea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe5890 0x55601cbe5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/131884040 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbe9ab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 -- 172.21.15.3:0/131884040 >> 172.21.15.3:0/131884040 conn(0x55601cb0d0b0 msgr2=0x55601cbdbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 -- 172.21.15.3:0/131884040 shutdown_connections 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.323+0000 7f9991b3fb80 1 -- 172.21.15.3:0/131884040 wait complete. 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 -- start start 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 Processor -- start 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbf14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbee840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe9ff0 0x55601cbeed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55601cbf2220 con 0x55601cbe61b0 2024-05-08T05:53:05.329 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55601ca99be0 con 0x55601cbe5890 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55601cbf23a0 con 0x55601cbe9ff0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9991b3fb80 10 client.0 _ll_register_callbacks cb 0x55601cacb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.324+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56182/0 (socket says 172.21.15.3:56182) 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 -- 172.21.15.3:0/4253154575 learned_addr learned my addr 172.21.15.3:0/4253154575 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9985048700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbf14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9985849700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe9ff0 0x55601cbeed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 -- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe9ff0 msgr2=0x55601cbeed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe9ff0 0x55601cbeed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 -- 172.21.15.3:0/4253154575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 msgr2=0x55601cbf14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbf14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 -- 172.21.15.3:0/4253154575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601cacf640 con 0x55601cbe61b0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9984847700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbee840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9970010450 tx=0x7f99700106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1994eda29b3339 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.330 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9968ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9968ff9700 1 -- 172.21.15.3:0/4253154575 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997006e020 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9968ff9700 1 -- 172.21.15.3:0/4253154575 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f99700b1ad0 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99700a8ef0 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9968ff9700 1 -- 172.21.15.3:0/4253154575 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f99700a95f0 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9991b3fb80 10 client.15021 fetch_fsmap learned FSMap version 21 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.325+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55601cbf2220 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9968ff9700 1 -- 172.21.15.3:0/4253154575 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99700a8ef0 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9968ff9700 1 -- 172.21.15.3:0/4253154575 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f99700a4a20 con 0x55601cbe61b0 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9991b3fb80 10 client.15021 fetch_fsmap finished waiting for FSMap version 21 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9991b3fb80 10 client.15021 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9991b3fb80 20 client.15021 populate_metadata read hostname 'smithi003' 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbfebf0 0x55601cbffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.331 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9991b3fb80 4 client.15021 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55601cbad600 con 0x55601cbfebf0 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.326+0000 7f9985048700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbfebf0 0x55601cbffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.327+0000 7f9985048700 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbfebf0 0x55601cbffff0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ea50dcb11f92ca server_cookie=becb8ce84ff4859b in_seq=0 out_seq=0 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.327+0000 7f9968ff9700 10 client.15021 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.327+0000 7f9968ff9700 1 -- 172.21.15.3:0/4253154575 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55601cbad600 con 0x55601cbfebf0 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.327+0000 7f9968ff9700 10 client.15021 handle_command_reply: tid=0 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.327+0000 7f9991b3fb80 1 client.15021 shutdown 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 msgr2=0x55601cbee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbee840 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9970010450 tx=0x7f99700106d0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 shutdown_connections 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe9ff0 0x55601cbeed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbfebf0 0x55601cbffff0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbf14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.332 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4253154575 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe61b0 0x55601cbee840 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9968ff9700 0 client.15021 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 >> 172.21.15.3:0/4253154575 conn(0x55601cb0d0b0 msgr2=0x55601cbd66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 shutdown_connections 2024-05-08T05:53:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4253154575 wait complete. 2024-05-08T05:53:05.333 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.328+0000 7f9991b3fb80 20 client.15021 trim_cache size 0 max 16384 2024-05-08T05:53:05.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.349+0000 7f9991b3fb80 1 Processor -- start 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.349+0000 7f9991b3fb80 1 -- start start 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cb0d0b0 0x55601cbeedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 0x55601cbec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cc09450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55601cbfe3a0 con 0x55601cbe5890 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55601cbfea70 con 0x55601cb0d0b0 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55601cbff0f0 con 0x55601cbe61b0 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 0x55601cbec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 0x55601cbec200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56194/0 (socket says 172.21.15.3:56194) 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 -- 172.21.15.3:0/4076932129 learned_addr learned my addr 172.21.15.3:0/4076932129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9985048700 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cb0d0b0 0x55601cbeedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9985849700 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cc09450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 -- 172.21.15.3:0/4076932129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 msgr2=0x55601cc09450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cc09450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 -- 172.21.15.3:0/4076932129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cb0d0b0 msgr2=0x55601cbeedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cb0d0b0 0x55601cbeedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 -- 172.21.15.3:0/4076932129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601cbfbd80 con 0x55601cbe5890 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9985849700 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cc09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.350+0000 7f9984847700 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 0x55601cbec200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99700a8e80 tx=0x7f9970004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb4bfd6de6be0e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f996affd700 1 -- 172.21.15.3:0/4076932129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997006e070 con 0x55601cbe5890 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f996affd700 1 -- 172.21.15.3:0/4076932129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99700a8ef0 con 0x55601cbe5890 2024-05-08T05:53:05.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f996affd700 1 -- 172.21.15.3:0/4076932129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99700a83b0 con 0x55601cbe5890 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4076932129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 msgr2=0x55601cbec200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 0x55601cbec200 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99700a8e80 tx=0x7f9970004860 comp rx=0 tx=0).stop 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4076932129 shutdown_connections 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cc09450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cb0d0b0 0x55601cbeedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/4076932129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cbe5890 0x55601cbec200 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.351+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4076932129 >> 172.21.15.3:0/4076932129 conn(0x55601cbd5a30 msgr2=0x55601cbebaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4076932129 shutdown_connections 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 -- 172.21.15.3:0/4076932129 wait complete. 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 -- start start 2024-05-08T05:53:05.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 Processor -- start 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 0x55601cbeb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbebd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cbec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55601cbffe70 con 0x55601cb0d0b0 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55601cbff7a0 con 0x55601cbe5890 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.352+0000 7f9991b3fb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55601cc00540 con 0x55601cbe61b0 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9991b3fb80 10 client.0 _ll_register_callbacks cb 0x55601cacb8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 0x55601cbeb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 0x55601cbeb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56196/0 (socket says 172.21.15.3:56196) 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 -- 172.21.15.3:0/1216008192 learned_addr learned my addr 172.21.15.3:0/1216008192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9984847700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbebd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985849700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cbec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 -- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 msgr2=0x55601cbec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cbec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 -- 172.21.15.3:0/1216008192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 msgr2=0x55601cbebd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbebd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 -- 172.21.15.3:0/1216008192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601cc07af0 con 0x55601cb0d0b0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9985048700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 0x55601cbeb7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55601cbf24b0 tx=0x7f997c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb1e76c94dd81d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9968ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9968ff9700 1 -- 172.21.15.3:0/1216008192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c032060 con 0x55601cb0d0b0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9968ff9700 1 -- 172.21.15.3:0/1216008192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f997c015a70 con 0x55601cb0d0b0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.353+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99700a8ef0 con 0x55601cb0d0b0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9968ff9700 1 -- 172.21.15.3:0/1216008192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f997c039050 con 0x55601cb0d0b0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 10 client.15027 fetch_fsmap learned FSMap version 21 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55601cc6a4a0 con 0x55601cb0d0b0 2024-05-08T05:53:05.358 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9968ff9700 1 -- 172.21.15.3:0/1216008192 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99700a8ef0 con 0x55601cb0d0b0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9968ff9700 1 -- 172.21.15.3:0/1216008192 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f997c02dc10 con 0x55601cb0d0b0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 10 client.15027 fetch_fsmap finished waiting for FSMap version 21 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 10 client.15027 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 20 client.15027 populate_metadata read hostname 'smithi003' 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbe9ff0 0x55601cc6ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 4 client.15027 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.354+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "status"}) -- 0x55601cb411f0 con 0x55601cbe9ff0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9984847700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbe9ff0 0x55601cc6ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9968ff9700 10 client.15027 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9984847700 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbe9ff0 0x55601cc6ddd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86747d2a79e8656 server_cookie=fa734ccbe5a46a01 in_seq=0 out_seq=0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9968ff9700 1 -- 172.21.15.3:0/1216008192 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55601cb411f0 con 0x55601cbe9ff0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9968ff9700 10 client.15027 handle_command_reply: tid=0 2024-05-08T05:53:05.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 client.15027 shutdown 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 msgr2=0x55601cbeb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 0x55601cbeb7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55601cbf24b0 tx=0x7f997c011000 comp rx=0 tx=0).stop 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 shutdown_connections 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55601cbe61b0 0x55601cbec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55601cb0d0b0 0x55601cbeb7e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55601cbe9ff0 0x55601cc6ddd0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 --2- 172.21.15.3:0/1216008192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55601cbe5890 0x55601cbebd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 >> 172.21.15.3:0/1216008192 conn(0x55601cbd5a30 msgr2=0x55601cbe0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9968ff9700 0 client.15027 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.355+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 shutdown_connections 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f9991b3fb80 1 -- 172.21.15.3:0/1216008192 wait complete. 2024-05-08T05:53:05.360 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f9991b3fb80 20 client.15027 trim_cache size 0 max 16384 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 182.71640707200001, 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 21, 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 89, 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2024-05-08T05:53:05.361 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 211.86892585300001 2024-05-08T05:53:05.362 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:53:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f996c0b2880 msgr2=0x7f996c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f996c0b2880 0x7f996c0b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f997c009a10 tx=0x7f997c009190 comp rx=0 tx=0).stop 2024-05-08T05:53:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 msgr2=0x7f998019b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:05.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998019b010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f997000b530 tx=0x7f997000b840 comp rx=0 tx=0).stop 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 shutdown_connections 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9980108410 0x7f99801a5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99801023a0 0x7f998019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f996c0b2880 0x7f996c0b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 --2- 172.21.15.3:0/1038148649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9980102d60 0x7f998019b010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 >> 172.21.15.3:0/1038148649 conn(0x7f9980079350 msgr2=0x7f9980101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 shutdown_connections 2024-05-08T05:53:05.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:05.356+0000 7f99872ac700 1 -- 172.21.15.3:0/1038148649 wait complete. 2024-05-08T05:53:06.170 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T05:53:06.170 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T05:53:06.180 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:53:06.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T05:53:06.435 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:06.889 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:06 smithi003 ceph-mon[30137]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:07.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:06 smithi089 ceph-mon[34495]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:07.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:06 smithi104 ceph-mon[34559]: pgmap v395: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.108+0000 7f8af031a700 1 -- 172.21.15.3:0/1079816023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101f30 msgr2=0x7f8ae810bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.110 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.108+0000 7f8af031a700 1 --2- 172.21.15.3:0/1079816023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101f30 0x7f8ae810bae0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ae000b0d0 tx=0x7f8ae000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 -- 172.21.15.3:0/1079816023 shutdown_connections 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 --2- 172.21.15.3:0/1079816023 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101f30 0x7f8ae810bae0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 --2- 172.21.15.3:0/1079816023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ae8104af0 0x7f8ae8104f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 --2- 172.21.15.3:0/1079816023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae8101570 0x7f8ae81019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 -- 172.21.15.3:0/1079816023 >> 172.21.15.3:0/1079816023 conn(0x7f8ae80fd130 msgr2=0x7f8ae80ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 -- 172.21.15.3:0/1079816023 shutdown_connections 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 -- 172.21.15.3:0/1079816023 wait complete. 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 Processor -- start 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 -- start start 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 0x7f8ae8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ae8101f30 0x7f8ae8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.111 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.109+0000 7f8af031a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae8104af0 0x7f8ae81a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8af031a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae810efb0 con 0x7f8ae8101570 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8af031a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae810ee30 con 0x7f8ae8104af0 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8af031a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae810f130 con 0x7f8ae8101f30 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 0x7f8ae8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 0x7f8ae8199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56208/0 (socket says 172.21.15.3:56208) 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 -- 172.21.15.3:0/1486488712 learned_addr learned my addr 172.21.15.3:0/1486488712 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aed8b5700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ae8101f30 0x7f8ae8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee8b7700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae8104af0 0x7f8ae81a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 -- 172.21.15.3:0/1486488712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ae8101f30 msgr2=0x7f8ae8199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ae8101f30 0x7f8ae8199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 -- 172.21.15.3:0/1486488712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae8104af0 msgr2=0x7f8ae81a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae8104af0 0x7f8ae81a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 -- 172.21.15.3:0/1486488712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae81a1290 con 0x7f8ae8101570 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8aee0b6700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 0x7f8ae8199630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400d9b0 tx=0x7f8ae400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:08.112 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4017070 con 0x7f8ae8101570 2024-05-08T05:53:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae4009de0 con 0x7f8ae8101570 2024-05-08T05:53:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae81a1580 con 0x7f8ae8101570 2024-05-08T05:53:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400f3c0 con 0x7f8ae8101570 2024-05-08T05:53:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.110+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae81a1a70 con 0x7f8ae8101570 2024-05-08T05:53:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.112+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae806a190 con 0x7f8ae8101570 2024-05-08T05:53:08.113 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.113+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f8ae4009930 con 0x7f8ae8101570 2024-05-08T05:53:08.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.114+0000 7f8adf7fe700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8ad40b2880 0x7f8ad40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:08.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.114+0000 7f8aed8b5700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8ad40b2880 0x7f8ad40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:08.114 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.114+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f8ae40e20e0 con 0x7f8ae8101570 2024-05-08T05:53:08.115 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.115+0000 7f8aed8b5700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8ad40b2880 0x7f8ad40b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806b0f0 tx=0x7f8ad800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:08.119 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.119+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ae40a9a10 con 0x7f8ae8101570 2024-05-08T05:53:08.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.350+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8ae80638e0 con 0x7f8ad40b2880 2024-05-08T05:53:08.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.352+0000 7f8adf7fe700 1 -- 172.21.15.3:0/1486488712 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8ae80638e0 con 0x7f8ad40b2880 2024-05-08T05:53:08.352 INFO:teuthology.orchestra.run.smithi003.stdout:Backend: cephadm 2024-05-08T05:53:08.352 INFO:teuthology.orchestra.run.smithi003.stdout:Available: Yes 2024-05-08T05:53:08.352 INFO:teuthology.orchestra.run.smithi003.stdout:Paused: No 2024-05-08T05:53:08.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8ad40b2880 msgr2=0x7f8ad40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8ad40b2880 0x7f8ad40b4d40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806b0f0 tx=0x7f8ad800b410 comp rx=0 tx=0).stop 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 msgr2=0x7f8ae8199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 0x7f8ae8199630 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ae400d9b0 tx=0x7f8ae400dcc0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 shutdown_connections 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8ae8104af0 0x7f8ae81a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f8ad40b2880 0x7f8ad40b4d40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8ae8101570 0x7f8ae8199630 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 --2- 172.21.15.3:0/1486488712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8ae8101f30 0x7f8ae8199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 >> 172.21.15.3:0/1486488712 conn(0x7f8ae80fd130 msgr2=0x7f8ae8107b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 shutdown_connections 2024-05-08T05:53:08.356 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:08.355+0000 7f8af031a700 1 -- 172.21.15.3:0/1486488712 wait complete. 2024-05-08T05:53:09.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:08 smithi089 ceph-mon[34495]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:09.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:08 smithi104 ceph-mon[34559]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:09.067 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:08 smithi003 ceph-mon[30137]: pgmap v396: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:09.098 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T05:53:09.354 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:09.763 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:09 smithi003 ceph-mon[30137]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:10.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:09 smithi089 ceph-mon[34495]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:10.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:09 smithi104 ceph-mon[34559]: from='client.15033 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:10.986 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:10 smithi003 ceph-mon[30137]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 -- 172.21.15.3:0/2552291642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea01068e0 msgr2=0x7f7ea0108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2552291642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea01068e0 0x7f7ea0108cd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009990 tx=0x7f7e94009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 -- 172.21.15.3:0/2552291642 shutdown_connections 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2552291642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea0109210 0x7f7ea010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2552291642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 0x7f7ea01063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2552291642 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea01068e0 0x7f7ea0108cd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 -- 172.21.15.3:0/2552291642 >> 172.21.15.3:0/2552291642 conn(0x7f7ea00fd9c0 msgr2=0x7f7ea00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.002+0000 7f7ea612a700 1 -- 172.21.15.3:0/2552291642 shutdown_connections 2024-05-08T05:53:11.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.003+0000 7f7ea612a700 1 -- 172.21.15.3:0/2552291642 wait complete. 2024-05-08T05:53:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.003+0000 7f7ea612a700 1 Processor -- start 2024-05-08T05:53:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.003+0000 7f7ea612a700 1 -- start start 2024-05-08T05:53:11.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.003+0000 7f7ea612a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 0x7f7ea0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7ea612a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea01068e0 0x7f7ea0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7ea612a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 0x7f7ea01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7ea612a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010f020 con 0x7f7ea0109210 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7ea612a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010eea0 con 0x7f7ea01068e0 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7ea612a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea010f1a0 con 0x7f7ea0103fb0 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 0x7f7ea01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 0x7f7ea0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 0x7f7ea01a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56228/0 (socket says 172.21.15.3:56228) 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 -- 172.21.15.3:0/2528537137 learned_addr learned my addr 172.21.15.3:0/2528537137 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9f7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 0x7f7ea0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:50028/0 (socket says 172.21.15.3:50028) 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9effd700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea01068e0 0x7f7ea0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:11.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 -- 172.21.15.3:0/2528537137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 msgr2=0x7f7ea0199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 0x7f7ea0199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 -- 172.21.15.3:0/2528537137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea01068e0 msgr2=0x7f7ea0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea01068e0 0x7f7ea0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 -- 172.21.15.3:0/2528537137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea01a12f0 con 0x7f7ea0109210 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9ffff700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 0x7f7ea01a0c50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7e98007ee0 tx=0x7f7e9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98004db0 con 0x7f7ea0109210 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e98005020 con 0x7f7ea0109210 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.004+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98015660 con 0x7f7ea0109210 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.005+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea01a15e0 con 0x7f7ea0109210 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.005+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea01a1ba0 con 0x7f7ea0109210 2024-05-08T05:53:11.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.005+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea006a190 con 0x7f7ea0109210 2024-05-08T05:53:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.007+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f7e9800db20 con 0x7f7ea0109210 2024-05-08T05:53:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.008+0000 7f7e9cff9700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7e880b2900 0x7f7e880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.008+0000 7f7e9f7fe700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7e880b2900 0x7f7e880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.008+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f7e980e2770 con 0x7f7ea0109210 2024-05-08T05:53:11.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.008+0000 7f7e9f7fe700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7e880b2900 0x7f7e880b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009a10 tx=0x7f7e90009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:11.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.011+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e98010070 con 0x7f7ea0109210 2024-05-08T05:53:11.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:10 smithi089 ceph-mon[34495]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:11.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:10 smithi104 ceph-mon[34559]: pgmap v397: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:11.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.234+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f7ea00638e0 con 0x7f7e880b2900 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.247+0000 7f7e9cff9700 1 -- 172.21.15.3:0/2528537137 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f7ea00638e0 con 0x7f7e880b2900 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.a smithi003 running (3m) 32s ago 3m 14.5M - 19.0.0-3538-gbee299f4 352576a2df49 a199896766ac 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.b smithi089 running (3m) 72s ago 3m 17.4M - 19.0.0-3538-gbee299f4 352576a2df49 6358f2ed2394 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.c smithi104 running (3m) 72s ago 3m 13.7M - 19.0.0-3538-gbee299f4 352576a2df49 505603efcfa7 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.d smithi003 running (3m) 32s ago 3m 15.1M - 19.0.0-3538-gbee299f4 352576a2df49 b97c61b19939 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.e smithi089 running (3m) 72s ago 3m 13.5M - 19.0.0-3538-gbee299f4 352576a2df49 770bed1fe319 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.f smithi104 running (3m) 72s ago 3m 13.8M - 19.0.0-3538-gbee299f4 352576a2df49 6bff09764f81 2024-05-08T05:53:11.248 INFO:teuthology.orchestra.run.smithi003.stdout:mds.g smithi003 running (3m) 32s ago 3m 14.4M - 19.0.0-3538-gbee299f4 352576a2df49 794a6320023f 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mds.h smithi089 running (3m) 72s ago 3m 15.9M - 19.0.0-3538-gbee299f4 352576a2df49 d0ff2251bc9e 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mds.i smithi104 running (3m) 72s ago 3m 26.3M - 19.0.0-3538-gbee299f4 352576a2df49 4bf57299307a 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mds.j smithi003 running (3m) 32s ago 3m 16.2M - 19.0.0-3538-gbee299f4 352576a2df49 3b7c68ae2cb4 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mds.k smithi089 running (4m) 72s ago 4m 16.1M - 19.0.0-3538-gbee299f4 352576a2df49 37dbc486ad7d 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mds.l smithi104 running (4m) 72s ago 4m 16.1M - 19.0.0-3538-gbee299f4 352576a2df49 58022b2dc0d3 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.x smithi003 *:9283,8765 running (11m) 32s ago 11m 532M - 19.0.0-3538-gbee299f4 352576a2df49 a09ad788764b 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.y smithi089 *:8443,8765 running (9m) 72s ago 9m 470M - 19.0.0-3538-gbee299f4 352576a2df49 bf127d7b5172 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mgr.z smithi104 *:8443,8765 running (9m) 72s ago 9m 468M - 19.0.0-3538-gbee299f4 352576a2df49 373bbb82490d 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mon.a smithi003 running (11m) 32s ago 11m 51.4M 2048M 19.0.0-3538-gbee299f4 352576a2df49 25a2aff035ee 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mon.b smithi089 running (9m) 72s ago 9m 40.1M 2048M 19.0.0-3538-gbee299f4 352576a2df49 2495a7dc70cb 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:mon.c smithi104 running (9m) 72s ago 9m 40.5M 2048M 19.0.0-3538-gbee299f4 352576a2df49 8f7693d5b131 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 smithi003 running (9m) 32s ago 9m 57.4M 4273M 19.0.0-3538-gbee299f4 352576a2df49 0310133f16c3 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 smithi089 running (8m) 72s ago 8m 46.3M 4272M 19.0.0-3538-gbee299f4 352576a2df49 2294fb74133e 2024-05-08T05:53:11.249 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 smithi104 running (8m) 72s ago 8m 57.0M 4272M 19.0.0-3538-gbee299f4 352576a2df49 54e755a3a40d 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 smithi003 running (7m) 32s ago 7m 55.5M 4273M 19.0.0-3538-gbee299f4 352576a2df49 1fe2b9afff9d 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 smithi089 running (7m) 72s ago 7m 45.8M 4272M 19.0.0-3538-gbee299f4 352576a2df49 9f757a5eba89 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 smithi104 running (7m) 72s ago 7m 57.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 d93814719372 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 smithi003 running (6m) 32s ago 6m 42.9M 4273M 19.0.0-3538-gbee299f4 352576a2df49 ae1fcecd0b0c 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 smithi089 running (6m) 72s ago 6m 53.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 f37446c68e26 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 smithi104 running (5m) 72s ago 5m 58.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 f6019baed9f8 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 smithi003 running (5m) 32s ago 5m 43.1M 4273M 19.0.0-3538-gbee299f4 352576a2df49 50577f7be600 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 smithi089 running (4m) 72s ago 4m 42.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 bb40211295ed 2024-05-08T05:53:11.250 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 smithi104 running (4m) 72s ago 4m 41.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 a44f00aeac32 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.250+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7e880b2900 msgr2=0x7f7e880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.250+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7e880b2900 0x7f7e880b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009a10 tx=0x7f7e90009190 comp rx=0 tx=0).stop 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 msgr2=0x7f7ea01a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 0x7f7ea01a0c50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7e98007ee0 tx=0x7f7e9800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 shutdown_connections 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f7e880b2900 0x7f7e880b4dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ea0109210 0x7f7ea01a0c50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ea0103fb0 0x7f7ea0199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.251 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 --2- 172.21.15.3:0/2528537137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ea01068e0 0x7f7ea0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 >> 172.21.15.3:0/2528537137 conn(0x7f7ea00fd9c0 msgr2=0x7f7ea0105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 shutdown_connections 2024-05-08T05:53:11.252 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:11.251+0000 7f7ea612a700 1 -- 172.21.15.3:0/2528537137 wait complete. 2024-05-08T05:53:11.919 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T05:53:12.172 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:12.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:12 smithi003 ceph-mon[30137]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:12.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:12 smithi003 ceph-mon[30137]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:12 smithi089 ceph-mon[34495]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:13.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:12 smithi089 ceph-mon[34495]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:12 smithi104 ceph-mon[34559]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:13.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:12 smithi104 ceph-mon[34559]: pgmap v398: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 -- 172.21.15.3:0/1186357563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0103fa0 msgr2=0x7f14e010a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 --2- 172.21.15.3:0/1186357563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0103fa0 0x7f14e010a850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b0d0 tx=0x7f14d800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:53:13.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 -- 172.21.15.3:0/1186357563 shutdown_connections 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 --2- 172.21.15.3:0/1186357563 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0103fa0 0x7f14e010a850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 --2- 172.21.15.3:0/1186357563 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14e0102400 0x7f14e0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 --2- 172.21.15.3:0/1186357563 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14e0103600 0x7f14e0103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 -- 172.21.15.3:0/1186357563 >> 172.21.15.3:0/1186357563 conn(0x7f14e00fd9a0 msgr2=0x7f14e00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 -- 172.21.15.3:0/1186357563 shutdown_connections 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 -- 172.21.15.3:0/1186357563 wait complete. 2024-05-08T05:53:13.887 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.886+0000 7f14e55a9700 1 Processor -- start 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 -- start start 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 0x7f14e0106390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14e0103600 0x7f14e01049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14e0103fa0 0x7f14e0104f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f14e010de90 con 0x7f14e0102400 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f14e010dd10 con 0x7f14e0103600 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14e55a9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14e010e010 con 0x7f14e0103fa0 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 0x7f14e0106390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:13.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 0x7f14e0106390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56270/0 (socket says 172.21.15.3:56270) 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 -- 172.21.15.3:0/108780625 learned_addr learned my addr 172.21.15.3:0/108780625 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14df7fe700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14e0103fa0 0x7f14e0104f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14de7fc700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14e0103600 0x7f14e01049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 -- 172.21.15.3:0/108780625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14e0103fa0 msgr2=0x7f14e0104f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14e0103fa0 0x7f14e0104f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 -- 172.21.15.3:0/108780625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14e0103600 msgr2=0x7f14e01049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14e0103600 0x7f14e01049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.887+0000 7f14deffd700 1 -- 172.21.15.3:0/108780625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e01057d0 con 0x7f14e0102400 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.888+0000 7f14deffd700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 0x7f14e0106390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d980 tx=0x7f14d000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.888+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0016070 con 0x7f14e0102400 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.888+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14d000be00 con 0x7f14e0102400 2024-05-08T05:53:13.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.888+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e01a37e0 con 0x7f14e0102400 2024-05-08T05:53:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.888+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d000f3c0 con 0x7f14e0102400 2024-05-08T05:53:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.888+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e01a3cf0 con 0x7f14e0102400 2024-05-08T05:53:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.889+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e006a190 con 0x7f14e0102400 2024-05-08T05:53:13.891 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.890+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f14d0019070 con 0x7f14e0102400 2024-05-08T05:53:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.891+0000 7f14c7fff700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14c80b2880 0x7f14c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.891+0000 7f14de7fc700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14c80b2880 0x7f14c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.891+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f14d00e29a0 con 0x7f14e0102400 2024-05-08T05:53:13.892 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.892+0000 7f14de7fc700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14c80b2880 0x7f14c80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14d400bbb0 tx=0x7f14d400b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:13.894 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:13.894+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14d0020020 con 0x7f14e0102400 2024-05-08T05:53:14.122 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.122+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f14e01a3f50 con 0x7f14c80b2880 2024-05-08T05:53:14.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.128+0000 7f14c7fff700 1 -- 172.21.15.3:0/108780625 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f14e01a3f50 con 0x7f14c80b2880 2024-05-08T05:53:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T05:53:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:mds.all 12/12 75s ago 4m smithi003=a;smithi003=d;smithi003=g;smithi003=j;smithi089=b;smithi089=e;smithi089=h;smithi089=k;smithi104=c;smithi104=f;smithi104=i;smithi104=l;count:12 2024-05-08T05:53:14.128 INFO:teuthology.orchestra.run.smithi003.stdout:mgr 3/3 75s ago 9m smithi003=x;smithi089=y;smithi104=z;count:3 2024-05-08T05:53:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:mon 3/3 75s ago 9m smithi003:172.21.15.3=a;smithi089:172.21.15.89=b;smithi104:172.21.15.104=c;count:3 2024-05-08T05:53:14.129 INFO:teuthology.orchestra.run.smithi003.stdout:osd 12 75s ago - 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14c80b2880 msgr2=0x7f14c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14c80b2880 0x7f14c80b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f14d400bbb0 tx=0x7f14d400b590 comp rx=0 tx=0).stop 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 msgr2=0x7f14e0106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 0x7f14e0106390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f14d000d980 tx=0x7f14d000dc90 comp rx=0 tx=0).stop 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 shutdown_connections 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14e0103fa0 0x7f14e0104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f14c80b2880 0x7f14c80b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:14.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f14e0102400 0x7f14e0106390 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 --2- 172.21.15.3:0/108780625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f14e0103600 0x7f14e01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 >> 172.21.15.3:0/108780625 conn(0x7f14e00fd9a0 msgr2=0x7f14e00ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 shutdown_connections 2024-05-08T05:53:14.132 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:14.131+0000 7f14e55a9700 1 -- 172.21.15.3:0/108780625 wait complete. 2024-05-08T05:53:14.780 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:14 smithi003 ceph-mon[30137]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:14.813 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T05:53:15.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:14 smithi089 ceph-mon[34495]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:15.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:14 smithi104 ceph-mon[34559]: pgmap v399: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:15.070 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:15.895 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:15 smithi003 ceph-mon[30137]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:15 smithi089 ceph-mon[34495]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:16.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:15 smithi104 ceph-mon[34559]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.780+0000 7f559110a700 1 -- 172.21.15.3:0/1427661157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c103fb0 msgr2=0x7f558c1063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.780+0000 7f559110a700 1 --2- 172.21.15.3:0/1427661157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c103fb0 0x7f558c1063a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5574009990 tx=0x7f5574009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:16.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.780+0000 7f559110a700 1 -- 172.21.15.3:0/1427661157 shutdown_connections 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.780+0000 7f559110a700 1 --2- 172.21.15.3:0/1427661157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f558c109210 0x7f558c10b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.780+0000 7f559110a700 1 --2- 172.21.15.3:0/1427661157 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c103fb0 0x7f558c1063a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.780+0000 7f559110a700 1 --2- 172.21.15.3:0/1427661157 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c1068e0 0x7f558c108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.781+0000 7f559110a700 1 -- 172.21.15.3:0/1427661157 >> 172.21.15.3:0/1427661157 conn(0x7f558c0fd9c0 msgr2=0x7f558c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.781+0000 7f559110a700 1 -- 172.21.15.3:0/1427661157 shutdown_connections 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.781+0000 7f559110a700 1 -- 172.21.15.3:0/1427661157 wait complete. 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.781+0000 7f559110a700 1 Processor -- start 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.781+0000 7f559110a700 1 -- start start 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f559110a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f558c103fb0 0x7f558c103910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f559110a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 0x7f558c101f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f559110a700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c109210 0x7f558c1024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f559110a700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10f020 con 0x7f558c1068e0 2024-05-08T05:53:16.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f559110a700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10eea0 con 0x7f558c109210 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f559110a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10f1a0 con 0x7f558c103fb0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 0x7f558c101f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 0x7f558c101f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56282/0 (socket says 172.21.15.3:56282) 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 -- 172.21.15.3:0/1642209953 learned_addr learned my addr 172.21.15.3:0/1642209953 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558b59e700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c109210 0x7f558c1024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558ad9d700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f558c103fb0 0x7f558c103910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 -- 172.21.15.3:0/1642209953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f558c103fb0 msgr2=0x7f558c103910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f558c103fb0 0x7f558c103910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 -- 172.21.15.3:0/1642209953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c109210 msgr2=0x7f558c1024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c109210 0x7f558c1024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 -- 172.21.15.3:0/1642209953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c102d50 con 0x7f558c1068e0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.782+0000 7f558a59c700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 0x7f558c101f60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c920 tx=0x7f557c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.783+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c007b00 con 0x7f558c1068e0 2024-05-08T05:53:16.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.783+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c004ce0 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.783+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c1a37f0 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.783+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557c005550 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.783+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c1a3d80 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.784+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c06a190 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.785+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f557c007ca0 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.786+0000 7f5583fff700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55780b2880 0x7f55780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.786+0000 7f558ad9d700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55780b2880 0x7f55780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.786+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f557c0e2510 con 0x7f558c1068e0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.786+0000 7f558ad9d700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55780b2880 0x7f55780b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f557400b5c0 tx=0x7f5574005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:16.802 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:16.789+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f557c014030 con 0x7f558c1068e0 2024-05-08T05:53:16.803 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:16 smithi003 ceph-mon[30137]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:17.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.009+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f558c0638e0 con 0x7f55780b2880 2024-05-08T05:53:17.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.011+0000 7f5583fff700 1 -- 172.21.15.3:0/1642209953 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f558c0638e0 con 0x7f55780b2880 2024-05-08T05:53:17.011 INFO:teuthology.orchestra.run.smithi003.stdout:HOST ADDR LABELS STATUS 2024-05-08T05:53:17.011 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 172.21.15.3 2024-05-08T05:53:17.011 INFO:teuthology.orchestra.run.smithi003.stdout:smithi089 172.21.15.89 2024-05-08T05:53:17.011 INFO:teuthology.orchestra.run.smithi003.stdout:smithi104 172.21.15.104 2024-05-08T05:53:17.011 INFO:teuthology.orchestra.run.smithi003.stdout:3 hosts in cluster 2024-05-08T05:53:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:16 smithi089 ceph-mon[34495]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.013+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55780b2880 msgr2=0x7f55780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.013+0000 7f559110a700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55780b2880 0x7f55780b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f557400b5c0 tx=0x7f5574005fd0 comp rx=0 tx=0).stop 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.013+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 msgr2=0x7f558c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.013+0000 7f559110a700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 0x7f558c101f60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c920 tx=0x7f557c00cc30 comp rx=0 tx=0).stop 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 shutdown_connections 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f558c109210 0x7f558c1024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:17.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f558c103fb0 0x7f558c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:17.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f55780b2880 0x7f55780b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:17.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 --2- 172.21.15.3:0/1642209953 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f558c1068e0 0x7f558c101f60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:17.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 >> 172.21.15.3:0/1642209953 conn(0x7f558c0fd9c0 msgr2=0x7f558c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:17.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 shutdown_connections 2024-05-08T05:53:17.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:17.014+0000 7f559110a700 1 -- 172.21.15.3:0/1642209953 wait complete. 2024-05-08T05:53:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:16 smithi104 ceph-mon[34559]: pgmap v400: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:17.805 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T05:53:18.063 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:18.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:18 smithi003 ceph-mon[30137]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:18.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:18 smithi003 ceph-mon[30137]: pgmap v401: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:19.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:18 smithi089 ceph-mon[34495]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:19.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:18 smithi089 ceph-mon[34495]: pgmap v401: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:18 smithi104 ceph-mon[34559]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:19.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:18 smithi104 ceph-mon[34559]: pgmap v401: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:19.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 -- 172.21.15.3:0/1210685686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520106920 msgr2=0x7f6520108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:19.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 --2- 172.21.15.3:0/1210685686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520106920 0x7f6520108d10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6514009990 tx=0x7f6514009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 -- 172.21.15.3:0/1210685686 shutdown_connections 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 --2- 172.21.15.3:0/1210685686 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6520109250 0x7f652010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 --2- 172.21.15.3:0/1210685686 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6520103ff0 0x7f65201063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 --2- 172.21.15.3:0/1210685686 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520106920 0x7f6520108d10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 -- 172.21.15.3:0/1210685686 >> 172.21.15.3:0/1210685686 conn(0x7f65200fd9a0 msgr2=0x7f65200ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 -- 172.21.15.3:0/1210685686 shutdown_connections 2024-05-08T05:53:19.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.739+0000 7f6526024700 1 -- 172.21.15.3:0/1210685686 wait complete. 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 Processor -- start 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 -- start start 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6520103ff0 0x7f6520103950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6520106920 0x7f6520101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 0x7f65201024e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f652010ef80 con 0x7f6520109250 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f652010ee00 con 0x7f6520103ff0 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f6526024700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f652010f100 con 0x7f6520106920 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f651ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 0x7f65201024e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f651ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 0x7f65201024e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56302/0 (socket says 172.21.15.3:56302) 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f651ffff700 1 -- 172.21.15.3:0/720983684 learned_addr learned my addr 172.21.15.3:0/720983684 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f651effd700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6520106920 0x7f6520101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:19.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.740+0000 7f651f7fe700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6520103ff0 0x7f6520103950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651ffff700 1 -- 172.21.15.3:0/720983684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6520106920 msgr2=0x7f6520101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651ffff700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6520106920 0x7f6520101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651ffff700 1 -- 172.21.15.3:0/720983684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6520103ff0 msgr2=0x7f6520103950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651ffff700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6520103ff0 0x7f6520103950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651ffff700 1 -- 172.21.15.3:0/720983684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6520102d90 con 0x7f6520109250 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651ffff700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 0x7f65201024e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f65180094f0 tx=0x7f651800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518004db0 con 0x7f6520109250 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65201a3810 con 0x7f6520109250 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6518009e20 con 0x7f6520109250 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518015400 con 0x7f6520109250 2024-05-08T05:53:19.743 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.741+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65201a3d20 con 0x7f6520109250 2024-05-08T05:53:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.743+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652006a190 con 0x7f6520109250 2024-05-08T05:53:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.744+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f6518005260 con 0x7f6520109250 2024-05-08T05:53:19.744 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.744+0000 7f651cff9700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f65080b2880 0x7f65080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:19.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.745+0000 7f651f7fe700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f65080b2880 0x7f65080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:19.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.745+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f65180e1f30 con 0x7f6520109250 2024-05-08T05:53:19.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.745+0000 7f651f7fe700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f65080b2880 0x7f65080b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6510007c60 tx=0x7f65100073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:19.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.749+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6518010070 con 0x7f6520109250 2024-05-08T05:53:19.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.973+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f652002e930 con 0x7f65080b2880 2024-05-08T05:53:19.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.975+0000 7f651cff9700 1 -- 172.21.15.3:0/720983684 <== mgr.14150 v2:172.21.15.3:6800/2793699832 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f652002e930 con 0x7f65080b2880 2024-05-08T05:53:19.975 INFO:teuthology.orchestra.run.smithi003.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T05:53:19.975 INFO:teuthology.orchestra.run.smithi003.stdout:smithi003 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009Z400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-08T05:53:19.976 INFO:teuthology.orchestra.run.smithi003.stdout:smithi089 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040126400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-08T05:53:19.976 INFO:teuthology.orchestra.run.smithi003.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f65080b2880 msgr2=0x7f65080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f65080b2880 0x7f65080b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6510007c60 tx=0x7f65100073d0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 msgr2=0x7f65201024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 0x7f65201024e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f65180094f0 tx=0x7f651800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 shutdown_connections 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f65080b2880 0x7f65080b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6520109250 0x7f65201024e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6520103ff0 0x7f6520103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.977+0000 7f6526024700 1 --2- 172.21.15.3:0/720983684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6520106920 0x7f6520101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:19.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.978+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 >> 172.21.15.3:0/720983684 conn(0x7f65200fd9a0 msgr2=0x7f6520105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:19.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.978+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 shutdown_connections 2024-05-08T05:53:19.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:19.978+0000 7f6526024700 1 -- 172.21.15.3:0/720983684 wait complete. 2024-05-08T05:53:20.677 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T05:53:20.686 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:53:20.686 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T05:53:20.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:20 smithi003 ceph-mon[30137]: pgmap v402: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:20.944 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:21.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:20 smithi089 ceph-mon[34495]: pgmap v402: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:21.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:20 smithi104 ceph-mon[34559]: pgmap v402: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:21 smithi089 ceph-mon[34495]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:21 smithi104 ceph-mon[34559]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:22.084 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:21 smithi003 ceph-mon[30137]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:22.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 -- 172.21.15.3:0/641972899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 msgr2=0x7f3fe0108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/641972899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0108d50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009990 tx=0x7f3fd4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 -- 172.21.15.3:0/641972899 shutdown_connections 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/641972899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0109290 0x7f3fe010b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/641972899 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe0104030 0x7f3fe0106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/641972899 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0108d50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 -- 172.21.15.3:0/641972899 >> 172.21.15.3:0/641972899 conn(0x7f3fe00fd9a0 msgr2=0x7f3fe00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:22.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 -- 172.21.15.3:0/641972899 shutdown_connections 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.595+0000 7f3fe75e0700 1 -- 172.21.15.3:0/641972899 wait complete. 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 Processor -- start 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 -- start start 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0104030 0x7f3fe0199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe0109290 0x7f3fe01a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe010f080 con 0x7f3fe0106960 2024-05-08T05:53:22.600 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe010ef00 con 0x7f3fe0109290 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe75e0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe010f200 con 0x7f3fe0104030 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe4b7b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe4b7b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56310/0 (socket says 172.21.15.3:56310) 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.596+0000 7f3fe4b7b700 1 -- 172.21.15.3:0/4225683990 learned_addr learned my addr 172.21.15.3:0/4225683990 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe537c700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0104030 0x7f3fe0199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe5b7d700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe0109290 0x7f3fe01a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe4b7b700 1 -- 172.21.15.3:0/4225683990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0104030 msgr2=0x7f3fe0199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe4b7b700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0104030 0x7f3fe0199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe4b7b700 1 -- 172.21.15.3:0/4225683990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe0109290 msgr2=0x7f3fe01a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe4b7b700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe0109290 0x7f3fe01a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.601 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe4b7b700 1 -- 172.21.15.3:0/4225683990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a13e0 con 0x7f3fe0106960 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe537c700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0104030 0x7f3fe0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe4b7b700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0199cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40038a0 tx=0x7f3fd40039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4027070 con 0x7f3fe0106960 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd4022470 con 0x7f3fe0106960 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a1670 con 0x7f3fe0106960 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400f6c0 con 0x7f3fe0106960 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.597+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a1ba0 con 0x7f3fe0106960 2024-05-08T05:53:22.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.598+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe006a190 con 0x7f3fe0106960 2024-05-08T05:53:22.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.601+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f3fd402a070 con 0x7f3fe0106960 2024-05-08T05:53:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.601+0000 7f3fd27fc700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3fcc0b2900 0x7f3fcc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.602+0000 7f3fe537c700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3fcc0b2900 0x7f3fcc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.602+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f3fd40e3960 con 0x7f3fe0106960 2024-05-08T05:53:22.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.602+0000 7f3fe537c700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3fcc0b2900 0x7f3fcc0b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0099d0 tx=0x7f3fdc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:22.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.605+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3fd4013070 con 0x7f3fe0106960 2024-05-08T05:53:22.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.867+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3fe002e510 con 0x7f3fe0106960 2024-05-08T05:53:22.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.868+0000 7f3fd27fc700 1 -- 172.21.15.3:0/4225683990 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) ==== 76+0+3002 (secure 0 0 0) 0x7f3fd40ab440 con 0x7f3fe0106960 2024-05-08T05:53:22.868 INFO:teuthology.orchestra.run.smithi003.stdout:e21 2024-05-08T05:53:22.868 INFO:teuthology.orchestra.run.smithi003.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:53:22.868 INFO:teuthology.orchestra.run.smithi003.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:legacy client fscid: 1 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:Filesystem 'cephfs' (1) 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:fs_name cephfs 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 21 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-05-08T05:50:01.809365+0000 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-05-08T05:51:05.493748+0000 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:tableserver 0 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:root 0 2024-05-08T05:53:22.869 INFO:teuthology.orchestra.run.smithi003.stdout:session_timeout 300 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:session_autoclose 300 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:max_file_size 1099511627776 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:max_xattr_size 65536 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:required_client_features {} 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure 0 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:last_failure_osd_epoch 0 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:max_mds 1 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:in 0 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:up {0=24493} 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:failed 2024-05-08T05:53:22.870 INFO:teuthology.orchestra.run.smithi003.stdout:damaged 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:stopped 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:data_pools [3,4] 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:metadata_pool 2 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:inline_data disabled 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:balancer 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:bal_rank_mask -1 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:standby_count_wanted 0 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:qdb_cluster leader: 24493 members: 24493 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.i{0:24493} state up:active seq 9 addr [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.b{0:24488} state up:standby-replay seq 1 addr [v2:172.21.15.89:6838/472688413,v1:172.21.15.89:6839/472688413] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:22.871 INFO:teuthology.orchestra.run.smithi003.stdout:Standby daemons: 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.j{-1:14499} state up:standby seq 1 addr [v2:172.21.15.3:6834/2960751422,v1:172.21.15.3:6835/2960751422] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.d{-1:14511} state up:standby seq 1 addr [v2:172.21.15.3:6836/2285392354,v1:172.21.15.3:6837/2285392354] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.g{-1:14559} state up:standby seq 1 addr [v2:172.21.15.3:6838/3875310227,v1:172.21.15.3:6839/3875310227] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.a{-1:14571} state up:standby seq 1 addr [v2:172.21.15.3:6840/1028446466,v1:172.21.15.3:6841/1028446466] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.k{-1:24416} state up:standby seq 1 addr [v2:172.21.15.89:6832/3884583981,v1:172.21.15.89:6833/3884583981] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.e{-1:24437} state up:standby seq 1 addr [v2:172.21.15.89:6834/3820045271,v1:172.21.15.89:6835/3820045271] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.104:6832/1254707053,v1:172.21.15.104:6833/1254707053] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.h{-1:24458} state up:standby seq 1 addr [v2:172.21.15.89:6836/1710942152,v1:172.21.15.89:6837/1710942152] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.f{-1:24475} state up:standby seq 1 addr [v2:172.21.15.104:6834/3213984653,v1:172.21.15.104:6835/3213984653] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.872 INFO:teuthology.orchestra.run.smithi003.stdout:[mds.c{-1:24481} state up:standby seq 1 addr [v2:172.21.15.104:6836/2023592505,v1:172.21.15.104:6837/2023592505] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3fcc0b2900 msgr2=0x7f3fcc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3fcc0b2900 0x7f3fcc0b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc0099d0 tx=0x7f3fdc008040 comp rx=0 tx=0).stop 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 msgr2=0x7f3fe0199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0199cc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40038a0 tx=0x7f3fd40039f0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 shutdown_connections 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3fe0109290 0x7f3fe01a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3fe0104030 0x7f3fe0199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f3fcc0b2900 0x7f3fcc0b4dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 --2- 172.21.15.3:0/4225683990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3fe0106960 0x7f3fe0199cc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:22.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 >> 172.21.15.3:0/4225683990 conn(0x7f3fe00fd9a0 msgr2=0x7f3fe0104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 shutdown_connections 2024-05-08T05:53:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:22.871+0000 7f3fe75e0700 1 -- 172.21.15.3:0/4225683990 wait complete. 2024-05-08T05:53:22.874 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:53:23.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:22 smithi089 ceph-mon[34495]: pgmap v403: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:23.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:22 smithi104 ceph-mon[34559]: pgmap v403: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:23.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:22 smithi003 ceph-mon[30137]: pgmap v403: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:23.615 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T05:53:23.874 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:23 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/4225683990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:53:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:23 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/4225683990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:53:24.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:23 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/4225683990' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:53:25.008 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:24 smithi003 ceph-mon[30137]: pgmap v404: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:25.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:24 smithi089 ceph-mon[34495]: pgmap v404: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:25.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:24 smithi104 ceph-mon[34559]: pgmap v404: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.544+0000 7fbee7135700 1 -- 172.21.15.3:0/1109581148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 msgr2=0x7fbee010b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.544+0000 7fbee7135700 1 --2- 172.21.15.3:0/1109581148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee010b970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b0d0 tx=0x7fbed800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- 172.21.15.3:0/1109581148 shutdown_connections 2024-05-08T05:53:25.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 --2- 172.21.15.3:0/1109581148 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee010b970 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 --2- 172.21.15.3:0/1109581148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0103520 0x7fbee0103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 --2- 172.21.15.3:0/1109581148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee0104720 0x7fbee0104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- 172.21.15.3:0/1109581148 >> 172.21.15.3:0/1109581148 conn(0x7fbee00feac0 msgr2=0x7fbee0100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- 172.21.15.3:0/1109581148 shutdown_connections 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- 172.21.15.3:0/1109581148 wait complete. 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 Processor -- start 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- start start 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0103520 0x7fbee0077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee0104720 0x7fbee0075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee0075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010efa0 con 0x7fbee01050c0 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010ee20 con 0x7fbee0103520 2024-05-08T05:53:25.548 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.545+0000 7fbee7135700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbee010f120 con 0x7fbee0104720 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee0075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee0075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34534/0 (socket says 172.21.15.3:34534) 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 -- 172.21.15.3:0/3321808447 learned_addr learned my addr 172.21.15.3:0/3321808447 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee4ed1700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0103520 0x7fbee0077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbedffff700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee0104720 0x7fbee0075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 -- 172.21.15.3:0/3321808447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee0104720 msgr2=0x7fbee0075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee0104720 0x7fbee0075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 -- 172.21.15.3:0/3321808447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0103520 msgr2=0x7fbee0077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0103520 0x7fbee0077240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 -- 172.21.15.3:0/3321808447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee0076680 con 0x7fbee01050c0 2024-05-08T05:53:25.549 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee56d2700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee0075dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009040 tx=0x7fbed8004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed801f030 con 0x7fbee01050c0 2024-05-08T05:53:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed8003a00 con 0x7fbee01050c0 2024-05-08T05:53:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee01a36c0 con 0x7fbee01050c0 2024-05-08T05:53:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.546+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed80046e0 con 0x7fbee01050c0 2024-05-08T05:53:25.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.547+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee01a3b10 con 0x7fbee01050c0 2024-05-08T05:53:25.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.547+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee006a190 con 0x7fbee01050c0 2024-05-08T05:53:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.549+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbed8003bb0 con 0x7fbee01050c0 2024-05-08T05:53:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.550+0000 7fbeddffb700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbed00b2900 0x7fbed00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.550+0000 7fbee4ed1700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbed00b2900 0x7fbed00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.550+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbed80e3700 con 0x7fbee01050c0 2024-05-08T05:53:25.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.551+0000 7fbee4ed1700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbed00b2900 0x7fbed00b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbee0104580 tx=0x7fbed400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:25.554 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.554+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbed80ab120 con 0x7fbee01050c0 2024-05-08T05:53:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.771+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbee00640b0 con 0x7fbee01050c0 2024-05-08T05:53:25.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.772+0000 7fbeddffb700 1 -- 172.21.15.3:0/3321808447 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) ==== 56+0+5619 (secure 0 0 0) 0x7fbed80aaa20 con 0x7fbee01050c0 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:epoch 89 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:fsid 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:created 2024-05-08T05:41:33.921408+0000 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:modified 2024-05-08T05:51:35.298219+0000 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:crush_version 27 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:full_ratio 0.95 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:backfillfull_ratio 0.9 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:nearfull_ratio 0.85 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:require_min_compat_client luminous 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:min_compat_client luminous 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:require_osd_release squid 2024-05-08T05:53:25.773 INFO:teuthology.orchestra.run.smithi003.stdout:stretch_mode_enabled false 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 78 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:max_osd 12 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6802/3730465717,v1:172.21.15.3:6803/3730465717] [v2:172.21.15.3:6804/3730465717,v1:172.21.15.3:6805/3730465717] exists,up 8a64e1a5-0f5b-4a16-b521-3b41c78d7269 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.1 up in weight 1 up_from 13 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6800/662806549,v1:172.21.15.89:6801/662806549] [v2:172.21.15.89:6802/662806549,v1:172.21.15.89:6803/662806549] exists,up 804db558-09e6-4d7d-9eca-b70cd45c71a8 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/3444157225,v1:172.21.15.104:6801/3444157225] [v2:172.21.15.104:6802/3444157225,v1:172.21.15.104:6803/3444157225] exists,up c61371f5-cc2f-45a6-88b2-eefe6596c9b5 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.3 up in weight 1 up_from 25 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6810/2348325148,v1:172.21.15.3:6811/2348325148] [v2:172.21.15.3:6812/2348325148,v1:172.21.15.3:6813/2348325148] exists,up 08946cb5-3b97-40ee-a729-6c9a386aeeb8 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.4 up in weight 1 up_from 31 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6808/3658777632,v1:172.21.15.89:6809/3658777632] [v2:172.21.15.89:6810/3658777632,v1:172.21.15.89:6811/3658777632] exists,up c7f59b73-f06d-4487-ad40-019eb1a6b470 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.5 up in weight 1 up_from 36 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/4290182422,v1:172.21.15.104:6809/4290182422] [v2:172.21.15.104:6810/4290182422,v1:172.21.15.104:6811/4290182422] exists,up 5d8e47b2-6008-416b-ba61-f08cf12933b9 2024-05-08T05:53:25.774 INFO:teuthology.orchestra.run.smithi003.stdout:osd.6 up in weight 1 up_from 42 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6818/1882383850,v1:172.21.15.3:6819/1882383850] [v2:172.21.15.3:6820/1882383850,v1:172.21.15.3:6821/1882383850] exists,up 7ace0899-0a39-4753-a710-047f463b5a7f 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6816/2376892451,v1:172.21.15.89:6817/2376892451] [v2:172.21.15.89:6818/2376892451,v1:172.21.15.89:6819/2376892451] exists,up 13aa9f3f-8b0f-43e4-b328-653822cc6b28 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:osd.8 up in weight 1 up_from 53 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/1691421157,v1:172.21.15.104:6817/1691421157] [v2:172.21.15.104:6818/1691421157,v1:172.21.15.104:6819/1691421157] exists,up a4924e7d-921a-4cbc-a59a-4b23e570e1c4 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:osd.9 up in weight 1 up_from 59 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.3:6826/1455470831,v1:172.21.15.3:6827/1455470831] [v2:172.21.15.3:6828/1455470831,v1:172.21.15.3:6829/1455470831] exists,up 99b34df5-382a-42ff-a9ed-12c6a02706a9 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:osd.10 up in weight 1 up_from 64 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.89:6824/3735647304,v1:172.21.15.89:6825/3735647304] [v2:172.21.15.89:6826/3735647304,v1:172.21.15.89:6827/3735647304] exists,up 4e939df5-46fd-4a17-ba2d-e3f72120e35e 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:osd.11 up in weight 1 up_from 70 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/1990686084,v1:172.21.15.104:6825/1990686084] [v2:172.21.15.104:6826/1990686084,v1:172.21.15.104:6827/1990686084] exists,up 262280c4-ebf6-4ea0-ba58-3c2eeab79323 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.4 [2,11] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.10 [2,4] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 2.1c [8,11] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.3 [4,9] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.e [5,9] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.1b [4,8] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.1d [4,6] 2024-05-08T05:53:25.775 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 3.29 [3,9] 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.2 [9,1] 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.12 [9,10] 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.19 [9,8] 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:pg_upmap_items 4.1d [5,8] 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/2743706157 expires 2024-05-09T05:42:33.122111+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/174548513 expires 2024-05-09T05:42:33.122111+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/174548513 expires 2024-05-09T05:42:33.122111+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6800/3412581798 expires 2024-05-09T05:42:06.081817+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/615172825 expires 2024-05-09T05:42:33.122111+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/4027937918 expires 2024-05-09T05:42:06.081817+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:6801/3412581798 expires 2024-05-09T05:42:06.081817+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/1221768244 expires 2024-05-09T05:42:06.081817+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/3389888238 expires 2024-05-09T05:42:06.081817+0000 2024-05-08T05:53:25.776 INFO:teuthology.orchestra.run.smithi003.stdout:blocklist 172.21.15.3:0/3812286739 expires 2024-05-09T05:42:33.122111+0000 2024-05-08T05:53:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbed00b2900 msgr2=0x7fbed00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbed00b2900 0x7fbed00b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbee0104580 tx=0x7fbed400b410 comp rx=0 tx=0).stop 2024-05-08T05:53:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 msgr2=0x7fbee0075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:25.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee0075dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009040 tx=0x7fbed8004d50 comp rx=0 tx=0).stop 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 shutdown_connections 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbed00b2900 0x7fbed00b4dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbee01050c0 0x7fbee0075dd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbee0103520 0x7fbee0077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 --2- 172.21.15.3:0/3321808447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbee0104720 0x7fbee0075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 >> 172.21.15.3:0/3321808447 conn(0x7fbee00feac0 msgr2=0x7fbee0107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 shutdown_connections 2024-05-08T05:53:25.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:25.775+0000 7fbee7135700 1 -- 172.21.15.3:0/3321808447 wait complete. 2024-05-08T05:53:26.522 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T05:53:26.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:26.790 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:27.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:26 smithi089 ceph-mon[34495]: pgmap v405: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:27.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:26 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3321808447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:53:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:26 smithi104 ceph-mon[34559]: pgmap v405: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:27.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:26 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3321808447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:53:27.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:26 smithi003 ceph-mon[30137]: pgmap v405: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:27.048 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:26 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3321808447' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 -- 172.21.15.3:0/3051272283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec103750 msgr2=0x7f53ec103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 --2- 172.21.15.3:0/3051272283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec103750 0x7f53ec103b30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f53dc0099c0 tx=0x7f53dc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 -- 172.21.15.3:0/3051272283 shutdown_connections 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 --2- 172.21.15.3:0/3051272283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 0x7f53ec1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 --2- 172.21.15.3:0/3051272283 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec103750 0x7f53ec103b30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 --2- 172.21.15.3:0/3051272283 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53ec104100 0x7f53ec108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 -- 172.21.15.3:0/3051272283 >> 172.21.15.3:0/3051272283 conn(0x7f53ec07a600 msgr2=0x7f53ec07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 -- 172.21.15.3:0/3051272283 shutdown_connections 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.534+0000 7f53f0906700 1 -- 172.21.15.3:0/3051272283 wait complete. 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 Processor -- start 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 -- start start 2024-05-08T05:53:28.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53ec103750 0x7f53ec0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 0x7f53ec075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 0x7f53ec076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec1136e0 con 0x7f53ec104100 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec113560 con 0x7f53ec103750 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53f0906700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec113860 con 0x7f53ec108850 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.535+0000 7f53e9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 0x7f53ec075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53e9d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 0x7f53ec075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34550/0 (socket says 172.21.15.3:34550) 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ea59c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53ec103750 0x7f53ec0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53e9d9b700 1 -- 172.21.15.3:0/812822679 learned_addr learned my addr 172.21.15.3:0/812822679 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 0x7f53ec076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 -- 172.21.15.3:0/812822679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53ec103750 msgr2=0x7f53ec0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53ec103750 0x7f53ec0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 -- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 msgr2=0x7f53ec075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 0x7f53ec075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 -- 172.21.15.3:0/812822679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1a80e0 con 0x7f53ec108850 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53e9d9b700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 0x7f53ec075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53ead9d700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 0x7f53ec076220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53e4007f10 tx=0x7f53e400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4004db0 con 0x7f53ec108850 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1a83d0 con 0x7f53ec108850 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53e4009e20 con 0x7f53ec108850 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.536+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4015400 con 0x7f53ec108850 2024-05-08T05:53:28.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.537+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec1a8960 con 0x7f53ec108850 2024-05-08T05:53:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.537+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec10ba90 con 0x7f53ec108850 2024-05-08T05:53:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.539+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f53e4005260 con 0x7f53ec108850 2024-05-08T05:53:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.540+0000 7f53db7fe700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53d40b2880 0x7f53d40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.540+0000 7f53ea59c700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53d40b2880 0x7f53d40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:28.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.540+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f53e40e23f0 con 0x7f53ec108850 2024-05-08T05:53:28.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.540+0000 7f53ea59c700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53d40b2880 0x7f53d40b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009990 tx=0x7f53dc005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:28.543 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.543+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53e4010070 con 0x7f53ec108850 2024-05-08T05:53:28.805 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.805+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53ec10b470 con 0x7f53ec108850 2024-05-08T05:53:28.806 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:28 smithi003 ceph-mon[30137]: pgmap v406: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:28.806 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.806+0000 7f53db7fe700 1 -- 172.21.15.3:0/812822679 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7f53e40a9d80 con 0x7f53ec108850 2024-05-08T05:53:28.807 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:28.807 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:53:28.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53d40b2880 msgr2=0x7f53d40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53d40b2880 0x7f53d40b4d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009990 tx=0x7f53dc005c40 comp rx=0 tx=0).stop 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 msgr2=0x7f53ec076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 0x7f53ec076220 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53e4007f10 tx=0x7f53e400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 shutdown_connections 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53ec108850 0x7f53ec076220 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f53ec103750 0x7f53ec0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f53d40b2880 0x7f53d40b4d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 --2- 172.21.15.3:0/812822679 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f53ec104100 0x7f53ec075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 >> 172.21.15.3:0/812822679 conn(0x7f53ec07a600 msgr2=0x7f53ec101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 shutdown_connections 2024-05-08T05:53:28.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:28.808+0000 7f53f0906700 1 -- 172.21.15.3:0/812822679 wait complete. 2024-05-08T05:53:28.810 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:53:29.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:28 smithi089 ceph-mon[34495]: pgmap v406: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:29.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:28 smithi104 ceph-mon[34559]: pgmap v406: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:29.634 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:29.897 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:29.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:29 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/812822679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:29 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/812822679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:29 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/812822679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:31.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:30 smithi104 ceph-mon[34559]: pgmap v407: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:31.054 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:30 smithi003 ceph-mon[30137]: pgmap v407: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:31.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:30 smithi089 ceph-mon[34495]: pgmap v407: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 -- 172.21.15.3:0/621048810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038103750 msgr2=0x7f9038103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 --2- 172.21.15.3:0/621048810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038103750 0x7f9038103b30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f90280099c0 tx=0x7f9028009cd0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 -- 172.21.15.3:0/621048810 shutdown_connections 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 --2- 172.21.15.3:0/621048810 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9038108850 0x7f90381103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 --2- 172.21.15.3:0/621048810 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038103750 0x7f9038103b30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 --2- 172.21.15.3:0/621048810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9038104100 0x7f9038108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 -- 172.21.15.3:0/621048810 >> 172.21.15.3:0/621048810 conn(0x7f903807a600 msgr2=0x7f903807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 -- 172.21.15.3:0/621048810 shutdown_connections 2024-05-08T05:53:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.601+0000 7f903c8b1700 1 -- 172.21.15.3:0/621048810 wait complete. 2024-05-08T05:53:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 Processor -- start 2024-05-08T05:53:31.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 -- start start 2024-05-08T05:53:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9038103750 0x7f903819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:31.604 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 0x7f903819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9038108850 0x7f90381999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f90381136c0 con 0x7f9038104100 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9038113540 con 0x7f9038103750 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903c8b1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9038113840 con 0x7f9038108850 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 0x7f903819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 0x7f903819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:34564/0 (socket says 172.21.15.3:34564) 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 -- 172.21.15.3:0/1385685391 learned_addr learned my addr 172.21.15.3:0/1385685391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9036d9d700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9038108850 0x7f90381999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f903659c700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9038103750 0x7f903819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 -- 172.21.15.3:0/1385685391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9038108850 msgr2=0x7f90381999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9038108850 0x7f90381999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.605 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 -- 172.21.15.3:0/1385685391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9038103750 msgr2=0x7f903819f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9038103750 0x7f903819f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.602+0000 7f9035d9b700 1 -- 172.21.15.3:0/1385685391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f903819a100 con 0x7f9038104100 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.603+0000 7f9035d9b700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 0x7f903819fd20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f902c00e9c0 tx=0x7f902c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.603+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c00cbd0 con 0x7f9038104100 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.603+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f903819a3f0 con 0x7f9038104100 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.603+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f902c00cd70 con 0x7f9038104100 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.603+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f902c005610 con 0x7f9038104100 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.603+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90381a66b0 con 0x7f9038104100 2024-05-08T05:53:31.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.604+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f903810ba90 con 0x7f9038104100 2024-05-08T05:53:31.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.605+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7f902c004750 con 0x7f9038104100 2024-05-08T05:53:31.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.606+0000 7f90277fe700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f90200b2880 0x7f90200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:31.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.606+0000 7f903659c700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f90200b2880 0x7f90200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:31.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.606+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f902c0e2af0 con 0x7f9038104100 2024-05-08T05:53:31.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.607+0000 7f903659c700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f90200b2880 0x7f90200b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9028000c00 tx=0x7f9028005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:31.609 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.609+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f902c054df0 con 0x7f9038104100 2024-05-08T05:53:31.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.874+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f903806a190 con 0x7f9038104100 2024-05-08T05:53:31.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.875+0000 7f90277fe700 1 -- 172.21.15.3:0/1385685391 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7f902c0aa480 con 0x7f9038104100 2024-05-08T05:53:31.875 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:31.876 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f90200b2880 msgr2=0x7f90200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f90200b2880 0x7f90200b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9028000c00 tx=0x7f9028005fd0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 msgr2=0x7f903819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 0x7f903819fd20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f902c00e9c0 tx=0x7f902c00ecd0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 shutdown_connections 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9038108850 0x7f90381999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9038103750 0x7f903819f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.878 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f90200b2880 0x7f90200b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 --2- 172.21.15.3:0/1385685391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9038104100 0x7f903819fd20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.877+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 >> 172.21.15.3:0/1385685391 conn(0x7f903807a600 msgr2=0x7f90381099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.878+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 shutdown_connections 2024-05-08T05:53:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:31.878+0000 7f903c8b1700 1 -- 172.21.15.3:0/1385685391 wait complete. 2024-05-08T05:53:31.879 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:53:32.598 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:53:32.861 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:32.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:32 smithi003 ceph-mon[30137]: pgmap v408: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:32.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:32 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1385685391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:32 smithi104 ceph-mon[34559]: pgmap v408: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:33.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:32 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1385685391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:32 smithi089 ceph-mon[34495]: pgmap v408: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:33.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:32 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1385685391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:53:34.594 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 -- 172.21.15.3:0/333879652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98108410 msgr2=0x7fee981087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 --2- 172.21.15.3:0/333879652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98108410 0x7fee981087f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fee94009990 tx=0x7fee94009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 -- 172.21.15.3:0/333879652 shutdown_connections 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 --2- 172.21.15.3:0/333879652 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee98102d60 0x7fee9810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 --2- 172.21.15.3:0/333879652 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98108410 0x7fee981087f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 --2- 172.21.15.3:0/333879652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee981023a0 0x7fee98102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 -- 172.21.15.3:0/333879652 >> 172.21.15.3:0/333879652 conn(0x7fee98079350 msgr2=0x7fee98079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 -- 172.21.15.3:0/333879652 shutdown_connections 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.592+0000 7fee9f41e700 1 -- 172.21.15.3:0/333879652 wait complete. 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 Processor -- start 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 -- start start 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee981023a0 0x7fee9819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.595 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 0x7fee9819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee98108410 0x7fee981a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fee981125b0 con 0x7fee98102d60 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fee98112430 con 0x7fee98108410 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9f41e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee98112730 con 0x7fee981023a0 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.593+0000 7fee9c9b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 0x7fee9819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 0x7fee9819c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53922/0 (socket says 172.21.15.3:53922) 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 -- 172.21.15.3:0/2579034192 learned_addr learned my addr 172.21.15.3:0/2579034192 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9d1ba700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee981023a0 0x7fee9819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9d9bb700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee98108410 0x7fee981a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 -- 172.21.15.3:0/2579034192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee981023a0 msgr2=0x7fee9819bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee981023a0 0x7fee9819bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 -- 172.21.15.3:0/2579034192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee98108410 msgr2=0x7fee981a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee98108410 0x7fee981a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 -- 172.21.15.3:0/2579034192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee981a0df0 con 0x7fee98102d60 2024-05-08T05:53:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9c9b9700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 0x7fee9819c320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee8800d950 tx=0x7fee8800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88016070 con 0x7fee98102d60 2024-05-08T05:53:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8800fd40 con 0x7fee98102d60 2024-05-08T05:53:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee981a10e0 con 0x7fee98102d60 2024-05-08T05:53:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8800e3b0 con 0x7fee98102d60 2024-05-08T05:53:34.597 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.594+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a8960 con 0x7fee98102d60 2024-05-08T05:53:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.595+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee9806a190 con 0x7fee98102d60 2024-05-08T05:53:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.597+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fee88019070 con 0x7fee98102d60 2024-05-08T05:53:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.597+0000 7fee8e7fc700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fee840b2900 0x7fee840b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.598+0000 7fee9d1ba700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fee840b2900 0x7fee840b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.598+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fee880e2350 con 0x7fee98102d60 2024-05-08T05:53:34.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.598+0000 7fee9d1ba700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fee840b2900 0x7fee840b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fee9400b5c0 tx=0x7fee94005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:34.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.601+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee880a9ce0 con 0x7fee98102d60 2024-05-08T05:53:34.825 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.825+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee981a8bc0 con 0x7fee98102d60 2024-05-08T05:53:34.826 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:34 smithi003 ceph-mon[30137]: pgmap v409: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:34.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.827+0000 7fee8e7fc700 1 -- 172.21.15.3:0/2579034192 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24666 (secure 0 0 0) 0x7fee880a95e0 con 0x7fee98102d60 2024-05-08T05:53:34.828 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:34.828 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":89,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:51:35.298219+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:53:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fee840b2900 msgr2=0x7fee840b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fee840b2900 0x7fee840b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fee9400b5c0 tx=0x7fee94005960 comp rx=0 tx=0).stop 2024-05-08T05:53:34.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 msgr2=0x7fee9819c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 0x7fee9819c320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fee8800d950 tx=0x7fee8800dc60 comp rx=0 tx=0).stop 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 shutdown_connections 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fee98108410 0x7fee981a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee981023a0 0x7fee9819bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fee840b2900 0x7fee840b4dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 --2- 172.21.15.3:0/2579034192 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fee98102d60 0x7fee9819c320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 >> 172.21.15.3:0/2579034192 conn(0x7fee98079350 msgr2=0x7fee9806b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 shutdown_connections 2024-05-08T05:53:34.831 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:34.830+0000 7fee9f41e700 1 -- 172.21.15.3:0/2579034192 wait complete. 2024-05-08T05:53:35.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:34 smithi104 ceph-mon[34559]: pgmap v409: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:35.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:34 smithi089 ceph-mon[34495]: pgmap v409: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:35.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:50:01.809365+0000', 'modified': '2024-05-08T05:51:05.493748+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6839/472688413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 472688413}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 472688413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.104:6839/3320987620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3320987620}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3320987620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24493]} 2024-05-08T05:53:35.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:53:35.486 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:53:35.743 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:36.115 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:35 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2579034192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:36.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:35 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2579034192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:36.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:35 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2579034192' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:53:36.869 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:36 smithi003 ceph-mon[30137]: pgmap v410: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:37.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:36 smithi089 ceph-mon[34495]: pgmap v410: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:37.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:36 smithi104 ceph-mon[34559]: pgmap v410: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 -- 172.21.15.3:0/536247929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa41023a0 msgr2=0x7fbfa4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/536247929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa41023a0 0x7fbfa4102820 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009990 tx=0x7fbf8c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 -- 172.21.15.3:0/536247929 shutdown_connections 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/536247929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfa4102d60 0x7fbfa410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/536247929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4108410 0x7fbfa41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/536247929 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa41023a0 0x7fbfa4102820 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 -- 172.21.15.3:0/536247929 >> 172.21.15.3:0/536247929 conn(0x7fbfa4079350 msgr2=0x7fbfa4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 -- 172.21.15.3:0/536247929 shutdown_connections 2024-05-08T05:53:37.426 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.424+0000 7fbfa90b1700 1 -- 172.21.15.3:0/536247929 wait complete. 2024-05-08T05:53:37.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.425+0000 7fbfa90b1700 1 Processor -- start 2024-05-08T05:53:37.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.425+0000 7fbfa90b1700 1 -- start start 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa90b1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfa41023a0 0x7fbfa419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa90b1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102d60 0x7fbfa419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa90b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 0x7fbfa41a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa90b1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4112530 con 0x7fbfa4108410 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa90b1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41123b0 con 0x7fbfa4102d60 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa90b1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa41126b0 con 0x7fbfa41023a0 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 0x7fbfa41a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 0x7fbfa41a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53942/0 (socket says 172.21.15.3:53942) 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 -- 172.21.15.3:0/2889643516 learned_addr learned my addr 172.21.15.3:0/2889643516 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfa41023a0 0x7fbfa419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102d60 0x7fbfa419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 -- 172.21.15.3:0/2889643516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfa41023a0 msgr2=0x7fbfa419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfa41023a0 0x7fbfa419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 -- 172.21.15.3:0/2889643516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102d60 msgr2=0x7fbfa419aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102d60 0x7fbfa419aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 -- 172.21.15.3:0/2889643516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa41a5f70 con 0x7fbfa4108410 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.426+0000 7fbfa359e700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 0x7fbfa41a5a30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c007ee0 tx=0x7fbf9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.427+0000 7fbfa0d99700 1 -- 172.21.15.3:0/2889643516 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00ddd0 con 0x7fbfa4108410 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.427+0000 7fbfa0d99700 1 -- 172.21.15.3:0/2889643516 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c004020 con 0x7fbfa4108410 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.427+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa41a61a0 con 0x7fbfa4108410 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.427+0000 7fbfa0d99700 1 -- 172.21.15.3:0/2889643516 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c015660 con 0x7fbfa4108410 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.427+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa41a66b0 con 0x7fbfa4108410 2024-05-08T05:53:37.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.429+0000 7fbfa0d99700 1 -- 172.21.15.3:0/2889643516 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fbf9c0041d0 con 0x7fbfa4108410 2024-05-08T05:53:37.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.430+0000 7fbfa0d99700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbf900b2880 0x7fbf900b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.430+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbf900b2880 0x7fbf900b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.430+0000 7fbfa0d99700 1 -- 172.21.15.3:0/2889643516 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbf9c0e1f90 con 0x7fbfa4108410 2024-05-08T05:53:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.430+0000 7fbfb3944b80 1 Processor -- start 2024-05-08T05:53:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.430+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbf900b2880 0x7fbf900b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.430+0000 7fbfb3944b80 1 -- start start 2024-05-08T05:53:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de78ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 0x55642de79430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55642ddc2810 con 0x55642de78ff0 2024-05-08T05:53:37.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55642dd61770 con 0x55642de751b0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55642ddbe7d0 con 0x55642de74890 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 0x55642de79430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 0x55642de79430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53948/0 (socket says 172.21.15.3:53948) 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 -- 172.21.15.3:0/2750480906 learned_addr learned my addr 172.21.15.3:0/2750480906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de78ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 -- 172.21.15.3:0/2750480906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 msgr2=0x55642de74c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de74c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 -- 172.21.15.3:0/2750480906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 msgr2=0x55642de78ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de78ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.431+0000 7fbfa359e700 1 -- 172.21.15.3:0/2750480906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55642dd0f880 con 0x55642de78ff0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.432+0000 7fbfa359e700 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 0x55642de79430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0a6490 tx=0x7fbf9c0a50e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d025b229751a087 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.432+0000 7fbf9959a700 1 -- 172.21.15.3:0/2750480906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0ad5d0 con 0x55642de78ff0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.432+0000 7fbf9959a700 1 -- 172.21.15.3:0/2750480906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c0b2080 con 0x55642de78ff0 2024-05-08T05:53:37.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.432+0000 7fbf9959a700 1 -- 172.21.15.3:0/2750480906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c004bd0 con 0x55642de78ff0 2024-05-08T05:53:37.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2750480906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 msgr2=0x55642de79430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.436 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 0x55642de79430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c0a6490 tx=0x7fbf9c0a50e0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2750480906 shutdown_connections 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de78ff0 0x55642de79430 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2750480906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de78ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2750480906 >> 172.21.15.3:0/2750480906 conn(0x55642dd9c0b0 msgr2=0x55642de6ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2750480906 shutdown_connections 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2750480906 wait complete. 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 -- start start 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.433+0000 7fbfb3944b80 1 Processor -- start 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de74890 0x55642de80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55642de81220 con 0x55642de751b0 2024-05-08T05:53:37.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55642dd28be0 con 0x55642de74890 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55642de813a0 con 0x55642de78ff0 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfb3944b80 10 client.0 _ll_register_callbacks cb 0x55642dd5a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53954/0 (socket says 172.21.15.3:53954) 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 -- 172.21.15.3:0/2366855629 learned_addr learned my addr 172.21.15.3:0/2366855629 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de74890 0x55642de80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 -- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 msgr2=0x55642de7dd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 -- 172.21.15.3:0/2366855629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de74890 msgr2=0x55642de80480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de74890 0x55642de80480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbf9a59c700 1 -- 172.21.15.3:0/2366855629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55642dd5e640 con 0x55642de751b0 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.434+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de74890 0x55642de80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7d820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009990 tx=0x7fbf8c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950b56dde2bd1a99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/2366855629 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c003a60 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/2366855629 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbf8c003c00 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf9c0b2080 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/2366855629 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf8c031050 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 10 client.15099 fetch_fsmap learned FSMap version 21 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55642de8ba80 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/2366855629 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf9c0b2080 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/2366855629 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf8c023590 con 0x55642de751b0 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 21 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 10 client.15099 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 20 client.15099 populate_metadata read hostname 'smithi003' 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.435+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642de8f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.436+0000 7fbfb3944b80 4 client.15099 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:53:37.439 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.436+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55642de3c600 con 0x55642de8e110 2024-05-08T05:53:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.436+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642de8f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.436+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642de8f580 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5184150026ee6c36 server_cookie=7a593c3c079fe316 in_seq=0 out_seq=0 2024-05-08T05:53:37.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.436+0000 7fbf8a7fc700 10 client.15099 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/2366855629 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55642de3c600 con 0x55642de8e110 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbf8a7fc700 10 client.15099 handle_command_reply: tid=0 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 client.15099 shutdown 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 msgr2=0x55642de7d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7d820 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c009990 tx=0x7fbf8c00bf90 comp rx=0 tx=0).stop 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 shutdown_connections 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642de8f580 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de74890 0x55642de80480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbf8a7fc700 0 client.15099 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:37.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/2366855629 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7d820 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.437+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 >> 172.21.15.3:0/2366855629 conn(0x55642dd9c0b0 msgr2=0x55642de656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.438+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 shutdown_connections 2024-05-08T05:53:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.438+0000 7fbfb3944b80 1 -- 172.21.15.3:0/2366855629 wait complete. 2024-05-08T05:53:37.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.438+0000 7fbfb3944b80 20 client.15099 trim_cache size 0 max 16384 2024-05-08T05:53:37.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 Processor -- start 2024-05-08T05:53:37.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 -- start start 2024-05-08T05:53:37.460 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de7c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de78ff0 0x55642de989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55642de8d8c0 con 0x55642de751b0 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55642de8df90 con 0x55642de78ff0 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55642de8e660 con 0x55642de74890 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.459+0000 7fbf9a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53962/0 (socket says 172.21.15.3:53962) 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 -- 172.21.15.3:0/3352648305 learned_addr learned my addr 172.21.15.3:0/3352648305 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de7c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbfa359e700 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de78ff0 0x55642de989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 -- 172.21.15.3:0/3352648305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 msgr2=0x55642de7c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de7c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 -- 172.21.15.3:0/3352648305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de78ff0 msgr2=0x55642de989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.461 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de78ff0 0x55642de989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 -- 172.21.15.3:0/3352648305 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55642de80950 con 0x55642de751b0 2024-05-08T05:53:37.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7c670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000c00 tx=0x7fbf8c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc35d70694c526c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf98d99700 1 -- 172.21.15.3:0/3352648305 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c039070 con 0x55642de751b0 2024-05-08T05:53:37.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf98d99700 1 -- 172.21.15.3:0/3352648305 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c028070 con 0x55642de751b0 2024-05-08T05:53:37.462 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.460+0000 7fbf98d99700 1 -- 172.21.15.3:0/3352648305 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0050c0 con 0x55642de751b0 2024-05-08T05:53:37.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3352648305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 msgr2=0x55642de7c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7c670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c000c00 tx=0x7fbf8c001e50 comp rx=0 tx=0).stop 2024-05-08T05:53:37.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3352648305 shutdown_connections 2024-05-08T05:53:37.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de78ff0 0x55642de989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.466 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de74890 0x55642de7c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3352648305 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de751b0 0x55642de7c670 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3352648305 >> 172.21.15.3:0/3352648305 conn(0x55642dd9c0b0 msgr2=0x55642de79990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3352648305 shutdown_connections 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3352648305 wait complete. 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.461+0000 7fbfb3944b80 1 -- start start 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 Processor -- start 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 0x55642de7d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de7d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55642de8f400 con 0x55642de74890 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55642de8ed30 con 0x55642de751b0 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55642de8fad0 con 0x55642de78ff0 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfb3944b80 10 client.0 _ll_register_callbacks cb 0x55642dd5a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:53:37.467 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 0x55642de7d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 0x55642de7d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53970/0 (socket says 172.21.15.3:53970) 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 -- 172.21.15.3:0/3471346920 learned_addr learned my addr 172.21.15.3:0/3471346920 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa359e700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de7d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 -- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 msgr2=0x55642de7db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 -- 172.21.15.3:0/3471346920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 msgr2=0x55642de7d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de7d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 -- 172.21.15.3:0/3471346920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55642de96ce0 con 0x55642de74890 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa359e700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbfa2d9d700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 0x55642de7d080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b6c0 tx=0x7fbf94013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22549d1b38dae151 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.462+0000 7fbf8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/3471346920 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf94004a30 con 0x55642de74890 2024-05-08T05:53:37.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8c028070 con 0x55642de74890 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/3471346920 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbf94004bd0 con 0x55642de74890 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/3471346920 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fbf94034050 con 0x55642de74890 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 10 client.15105 fetch_fsmap learned FSMap version 21 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55642def98d0 con 0x55642de74890 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/3471346920 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf8c028070 con 0x55642de74890 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/3471346920 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7fbf94028ce0 con 0x55642de74890 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 10 client.15105 fetch_fsmap finished waiting for FSMap version 21 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 10 client.15105 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 20 client.15105 populate_metadata read hostname 'smithi003' 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642defd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 4 client.15105 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.463+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "status"}) -- 0x55642ddd01f0 con 0x55642de8e110 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642defd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbf9a59c700 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642defd270 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5366c6e3b3aaa83 server_cookie=e4af7e8de880ba1c in_seq=0 out_seq=0 2024-05-08T05:53:37.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbf8a7fc700 10 client.15105 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbf8a7fc700 1 -- 172.21.15.3:0/3471346920 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55642ddd01f0 con 0x55642de8e110 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbf8a7fc700 10 client.15105 handle_command_reply: tid=0 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 client.15105 shutdown 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 msgr2=0x55642de7d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 0x55642de7d080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400b6c0 tx=0x7fbf94013070 comp rx=0 tx=0).stop 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 shutdown_connections 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55642de78ff0 0x55642de7db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55642de74890 0x55642de7d080 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.464+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55642de8e110 0x55642defd270 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfb3944b80 1 --2- 172.21.15.3:0/3471346920 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55642de751b0 0x55642de7d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 >> 172.21.15.3:0/3471346920 conn(0x55642dd9c0b0 msgr2=0x55642de7abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbf8a7fc700 0 client.15105 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:53:37.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 shutdown_connections 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfb3944b80 1 -- 172.21.15.3:0/3471346920 wait complete. 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfb3944b80 20 client.15105 trim_cache size 0 max 16384 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-08T05:53:37.471 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 214.82562683200001, 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 21, 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 89, 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 87, 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 243.97814521800001 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbf900b2880 msgr2=0x7fbf900b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbf900b2880 0x7fbf900b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf94005950 tx=0x7fbf9400b410 comp rx=0 tx=0).stop 2024-05-08T05:53:37.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 msgr2=0x7fbfa41a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 0x7fbfa41a5a30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c007ee0 tx=0x7fbf9c00d3b0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 shutdown_connections 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbf900b2880 0x7fbf900b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbfa4108410 0x7fbfa41a5a30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfa41023a0 0x7fbfa419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 --2- 172.21.15.3:0/2889643516 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbfa4102d60 0x7fbfa419aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.465+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 >> 172.21.15.3:0/2889643516 conn(0x7fbfa4079350 msgr2=0x7fbfa4100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.466+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 shutdown_connections 2024-05-08T05:53:37.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:37.466+0000 7fbfa90b1700 1 -- 172.21.15.3:0/2889643516 wait complete. 2024-05-08T05:53:38.151 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:37 smithi003 ceph-mon[30137]: pgmap v411: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:38.181 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T05:53:38.191 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T05:53:38.191 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T05:53:38.192 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T05:53:38.192 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-08T05:53:38.193 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:53:38.193 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:53:38.193 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-08T05:53:38.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:37 smithi089 ceph-mon[34495]: pgmap v411: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:38.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:37 smithi104 ceph-mon[34559]: pgmap v411: 129 pgs: 129 active+clean; 641 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 -- 172.21.15.3:0/2724646721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 msgr2=0x7feca4102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 --2- 172.21.15.3:0/2724646721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca4102f80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fec94009ca0 tx=0x7fec9402a8c0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 -- 172.21.15.3:0/2724646721 shutdown_connections 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 --2- 172.21.15.3:0/2724646721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feca41034c0 0x7feca4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 --2- 172.21.15.3:0/2724646721 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca4102f80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 --2- 172.21.15.3:0/2724646721 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca4101910 0x7feca4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 -- 172.21.15.3:0/2724646721 >> 172.21.15.3:0/2724646721 conn(0x7feca40fce80 msgr2=0x7feca40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 -- 172.21.15.3:0/2724646721 shutdown_connections 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.303+0000 7fecaac83640 1 -- 172.21.15.3:0/2724646721 wait complete. 2024-05-08T05:53:38.306 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 Processor -- start 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 -- start start 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca4101910 0x7feca4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feca41034c0 0x7feca41a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7feca410e5c0 con 0x7feca4102b00 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7feca410e440 con 0x7feca4101910 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7fecaac83640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feca410e740 con 0x7feca41034c0 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7feca3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7feca3fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca41991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53994/0 (socket says 172.21.15.3:53994) 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7feca3fff640 1 -- 172.21.15.3:0/1955007469 learned_addr learned my addr 172.21.15.3:0/1955007469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7feca9220640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feca41034c0 0x7feca41a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.304+0000 7feca8a1f640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca4101910 0x7feca4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:38.307 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca3fff640 1 -- 172.21.15.3:0/1955007469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feca41034c0 msgr2=0x7feca41a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca3fff640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feca41034c0 0x7feca41a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca3fff640 1 -- 172.21.15.3:0/1955007469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca4101910 msgr2=0x7feca4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca3fff640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca4101910 0x7feca4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca3fff640 1 -- 172.21.15.3:0/1955007469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca41a08c0 con 0x7feca4102b00 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca3fff640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca41991c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec94000f80 tx=0x7fec94003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9402f380 con 0x7feca4102b00 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41a0b50 con 0x7feca4102b00 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec940317c0 con 0x7feca4102b00 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9402fbe0 con 0x7feca4102b00 2024-05-08T05:53:38.308 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.305+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca41a10a0 con 0x7feca4102b00 2024-05-08T05:53:38.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.306+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca4108c50 con 0x7feca4102b00 2024-05-08T05:53:38.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.309+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 18) ==== 149651+0+0 (secure 0 0 0) 0x7fec9402fd80 con 0x7feca4102b00 2024-05-08T05:53:38.309 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.309+0000 7feca1ffb640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fec900b27e0 0x7fec900b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:38.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.310+0000 7feca8a1f640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fec900b27e0 0x7fec900b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:38.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.310+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fec94103b30 con 0x7feca4102b00 2024-05-08T05:53:38.310 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.310+0000 7feca8a1f640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fec900b27e0 0x7fec900b4ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feca4199b50 tx=0x7fec8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:38.312 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.312+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec940cb4c0 con 0x7feca4102b00 2024-05-08T05:53:38.556 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.556+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7feca402dd50 con 0x7feca4102b00 2024-05-08T05:53:38.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.789+0000 7feca1ffb640 1 -- 172.21.15.3:0/1955007469 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fec940cadc0 con 0x7feca4102b00 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fec900b27e0 msgr2=0x7fec900b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fec900b27e0 0x7fec900b4ca0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feca4199b50 tx=0x7fec8c00b390 comp rx=0 tx=0).stop 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 msgr2=0x7feca41991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca41991c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec94000f80 tx=0x7fec94003ba0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 shutdown_connections 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fec900b27e0 0x7fec900b4ca0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feca41034c0 0x7feca41a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7feca4102b00 0x7feca41991c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 --2- 172.21.15.3:0/1955007469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7feca4101910 0x7feca4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.791+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 >> 172.21.15.3:0/1955007469 conn(0x7feca40fce80 msgr2=0x7feca4105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.792+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 shutdown_connections 2024-05-08T05:53:38.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:38.792+0000 7fecaac83640 1 -- 172.21.15.3:0/1955007469 wait complete. 2024-05-08T05:53:38.812 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T05:53:38.826 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T05:53:38.827 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T05:53:38.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:38.827 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-08T05:53:38.846 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T05:53:38.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:38.847 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:53:38.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:38.914 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-05-08T05:53:39.032 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-05-08T05:53:39.032 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:53:39.032 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:53:39.032 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T05:53:39.039 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:38 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1955007469' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T05:53:39.039 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:38 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: ignoring --setuser ceph since I am not root 2024-05-08T05:53:39.039 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:38 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: ignoring --setgroup ceph since I am not root 2024-05-08T05:53:39.039 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:39.011+0000 7fb936aa7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:53:39.151 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:38 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1955007469' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T05:53:39.152 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: ignoring --setuser ceph since I am not root 2024-05-08T05:53:39.152 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: ignoring --setgroup ceph since I am not root 2024-05-08T05:53:39.152 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:38 smithi104 ceph-mgr[36385]: -- 172.21.15.104:0/1862690544 <== mon.0 v2:172.21.15.3:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c271145a0 con 0x558c270e9000 2024-05-08T05:53:39.152 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:38.969+0000 7fce6f838200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:53:39.152 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:39.151+0000 7fce6f838200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:53:39.152 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:38 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1955007469' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T05:53:39.153 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:38 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: ignoring --setuser ceph since I am not root 2024-05-08T05:53:39.153 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:38 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: ignoring --setgroup ceph since I am not root 2024-05-08T05:53:39.153 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:38 smithi089 ceph-mgr[35921]: -- 172.21.15.89:0/3599546490 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fac34285a0 con 0x55fac33fc800 2024-05-08T05:53:39.153 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:38 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:38.967+0000 7f5db1f96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:53:39.153 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:39.151+0000 7f5db1f96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:53:39.293 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:39.345 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:39.215+0000 7fb936aa7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:53:39.345 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:39.345+0000 7fb936aa7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:53:39.512 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:39.254+0000 7f5db1f96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:53:39.520 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:39.254+0000 7fce6f838200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:53:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:39 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1955007469' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T05:53:40.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:39 smithi089 ceph-mon[34495]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-08T05:53:40.013 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:39.561+0000 7f5db1f96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:53:40.013 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:39.646+0000 7f5db1f96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:53:40.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:39 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1955007469' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T05:53:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:39 smithi104 ceph-mon[34559]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-08T05:53:40.021 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:39.560+0000 7fce6f838200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:53:40.021 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:39.644+0000 7fce6f838200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:53:40.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:39 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1955007469' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T05:53:40.152 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:39 smithi003 ceph-mon[30137]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-08T05:53:40.152 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:39.672+0000 7fb936aa7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:53:40.152 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:39.766+0000 7fb936aa7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:53:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 -- 172.21.15.3:0/962123639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1198108410 msgr2=0x7f11981087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 --2- 172.21.15.3:0/962123639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1198108410 0x7f11981087f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1188009990 tx=0x7f1188009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 -- 172.21.15.3:0/962123639 shutdown_connections 2024-05-08T05:53:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 --2- 172.21.15.3:0/962123639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f119810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 --2- 172.21.15.3:0/962123639 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1198108410 0x7f11981087f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 --2- 172.21.15.3:0/962123639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f11981023a0 0x7f1198102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 -- 172.21.15.3:0/962123639 >> 172.21.15.3:0/962123639 conn(0x7f1198079350 msgr2=0x7f1198079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 -- 172.21.15.3:0/962123639 shutdown_connections 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.020+0000 7f119cb45700 1 -- 172.21.15.3:0/962123639 wait complete. 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 Processor -- start 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 -- start start 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11981023a0 0x7f1198104400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f1198104940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1198108410 0x7f1198104e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1198112560 con 0x7f11981023a0 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f11981123e0 con 0x7f1198108410 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.021+0000 7f119cb45700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11981126e0 con 0x7f1198102d60 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f1198104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f119659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11981023a0 0x7f1198104400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f1198104940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:49210/0 (socket says 172.21.15.3:49210) 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 -- 172.21.15.3:0/987898797 learned_addr learned my addr 172.21.15.3:0/987898797 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f1196d9d700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1198108410 0x7f1198104e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 -- 172.21.15.3:0/987898797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1198108410 msgr2=0x7f1198104e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1198108410 0x7f1198104e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 -- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11981023a0 msgr2=0x7f1198104400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11981023a0 0x7f1198104400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 -- 172.21.15.3:0/987898797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11981a80c0 con 0x7f1198102d60 2024-05-08T05:53:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118dd9b700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f1198104940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9a0 tx=0x7f118000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:41.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118000ca10 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11981a83b0 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118000cbb0 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180018680 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.022+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11981a8940 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.023+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119806a190 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.025+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f118004aac0 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.025+0000 7f118ffff700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 0x7f11840b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.025+0000 7f119659c700 1 -- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 msgr2=0x7f11840b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.026+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f11800e2820 con 0x7f1198102d60 2024-05-08T05:53:41.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.026+0000 7f119659c700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 0x7f11840b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:53:41.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.029+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11800aa150 con 0x7f1198102d60 2024-05-08T05:53:41.158 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:40 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:40.817+0000 7f5db1f96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:53:41.158 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:40 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:40.989+0000 7f5db1f96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:53:41.158 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:41.158+0000 7f5db1f96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:53:41.166 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:40 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:40.824+0000 7fce6f838200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:53:41.166 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:40 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:40.995+0000 7fce6f838200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:53:41.166 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:41.166+0000 7fce6f838200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:53:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.227+0000 7f119659c700 1 -- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 msgr2=0x7f11840b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:41.227 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.227+0000 7f119659c700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 0x7f11840b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:53:41.295 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.295+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f11981a8ca0 con 0x7f1198102d60 2024-05-08T05:53:41.296 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.295+0000 7f118ffff700 1 -- 172.21.15.3:0/987898797 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) ==== 53+0+88 (secure 0 0 0) 0x7f11800b21d0 con 0x7f1198102d60 2024-05-08T05:53:41.296 INFO:teuthology.orchestra.run.smithi003.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:53:41.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 msgr2=0x7f11840b4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:53:41.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 0x7f11840b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.298 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 msgr2=0x7f1198104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f1198104940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9a0 tx=0x7f118000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 shutdown_connections 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1198108410 0x7f1198104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f11840b2950 0x7f11840b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f11981023a0 0x7f1198104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 --2- 172.21.15.3:0/987898797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1198102d60 0x7f1198104940 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 >> 172.21.15.3:0/987898797 conn(0x7f1198079350 msgr2=0x7f1198106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 shutdown_connections 2024-05-08T05:53:41.299 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:41.298+0000 7f119cb45700 1 -- 172.21.15.3:0/987898797 wait complete. 2024-05-08T05:53:41.311 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:41.056+0000 7fb936aa7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:53:41.311 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:41.238+0000 7fb936aa7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:53:41.448 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:41 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/987898797' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:53:41.456 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:41 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/987898797' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:53:41.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:41 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/987898797' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:53:41.660 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:41.422+0000 7fb936aa7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:53:41.741 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:41.447+0000 7f5db1f96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:53:41.741 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:41.539+0000 7f5db1f96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:53:41.752 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:41.456+0000 7fce6f838200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:53:41.752 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:41.550+0000 7fce6f838200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:53:41.752 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:41.751+0000 7fce6f838200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:53:42.000 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:41.730+0000 7fb936aa7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:53:42.001 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:41.828+0000 7fb936aa7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:53:42.012 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:41.740+0000 7f5db1f96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:53:42.012 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:41.935+0000 7f5db1f96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:53:42.020 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:41.944+0000 7fce6f838200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi003.front.sepia.ceph.com 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:53:42.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:53:42.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:53:42.031 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:53:42.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:42.031 DEBUG:teuthology.orchestra.run.smithi003:> ip addr 2024-05-08T05:53:42.087 INFO:teuthology.orchestra.run.smithi003.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:53:42.087 INFO:teuthology.orchestra.run.smithi003.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:53:42.087 INFO:teuthology.orchestra.run.smithi003.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:53:42.087 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 ::1/128 scope host 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ac brd ff:ff:ff:ff:ff:ff 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f0 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:6b:fd:ad brd ff:ff:ff:ff:ff:ff 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp7s0f1 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dc brd ff:ff:ff:ff:ff:ff 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f0 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: inet 172.21.15.3/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft 41966sec preferred_lft 41966sec 2024-05-08T05:53:42.088 INFO:teuthology.orchestra.run.smithi003.stdout: inet6 fe80::ec4:7aff:febd:15dc/64 scope link noprefixroute 2024-05-08T05:53:42.089 INFO:teuthology.orchestra.run.smithi003.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:53:42.089 INFO:teuthology.orchestra.run.smithi003.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:53:42.089 INFO:teuthology.orchestra.run.smithi003.stdout: link/ether 0c:c4:7a:bd:15:dd brd ff:ff:ff:ff:ff:ff 2024-05-08T05:53:42.089 INFO:teuthology.orchestra.run.smithi003.stdout: altname enp1s0f1 2024-05-08T05:53:42.089 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:53:42.089 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:53:42.089 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:53:42.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:53:42.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr flush dev ceph-brx 2024-05-08T05:53:42.089 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set ceph-brx up 2024-05-08T05:53:42.090 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:53:42.090 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:53:42.179 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:53:42.230 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:42.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:42.240 DEBUG:teuthology.orchestra.run.smithi003:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:53:42.261 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:42.044+0000 7fb936aa7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:53:42.266 INFO:teuthology.orchestra.run.smithi003.stdout:1 2024-05-08T05:53:42.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:42.268 DEBUG:teuthology.orchestra.run.smithi003:> route 2024-05-08T05:53:42.326 INFO:teuthology.orchestra.run.smithi003.stdout:Kernel IP routing table 2024-05-08T05:53:42.326 INFO:teuthology.orchestra.run.smithi003.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:53:42.326 INFO:teuthology.orchestra.run.smithi003.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:53:42.326 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:53:42.326 INFO:teuthology.orchestra.run.smithi003.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:53:42.327 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:53:42.327 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:53:42.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:53:42.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:53:42.327 DEBUG:teuthology.orchestra.run.smithi003:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:53:42.327 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:53:42.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:53:42.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:42.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:42.459 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-08T05:53:42.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:42.476 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list-id 2024-05-08T05:53:42.533 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:53:42.533 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:53:42.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:53:42.533 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T05:53:42.533 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:53:42.625 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:53:42.633 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:42.260+0000 7fb936aa7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:53:42.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:42.657 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T05:53:42.657 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:53:42.708 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:53:42.775 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:42.781 DEBUG:teuthology.orchestra.run.smithi003:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:53:42.781 DEBUG:teuthology.orchestra.run.smithi003:> set -e 2024-05-08T05:53:42.781 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set brx.0 up 2024-05-08T05:53:42.782 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:53:42.782 DEBUG:teuthology.orchestra.run.smithi003:> ') 2024-05-08T05:53:42.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:53:42.859 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:42.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:53:42.865 DEBUG:teuthology.orchestra.run.smithi003:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:53:42.881 INFO:teuthology.orchestra.run.smithi003.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:53:42.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:53:42.881 DEBUG:teuthology.orchestra.run.smithi003:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:53:42.937 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T05:53:42.937 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T05:53:42.937 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/95b7cec1-e786-453a-8829-7f21d652d298 2024-05-08T05:53:42.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:42.937 DEBUG:teuthology.orchestra.run.smithi003:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/95b7cec1-e786-453a-8829-7f21d652d298 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-08T05:53:43.012 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:42 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:42.532+0000 7f5db1f96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:53:43.012 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:42 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:42.750+0000 7f5db1f96200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:53:43.020 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:42.540+0000 7fce6f838200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:53:43.021 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:42.758+0000 7fce6f838200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:53:43.022 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-08T05:53:43.022 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-08T05:53:43.151 INFO:teuthology.orchestra.run.smithi003.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-05-08T05:53:43.151 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-05-08T05:53:43.151 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: resolved to: "172.21.15.3:6789,172.21.15.89:6789,172.21.15.104:6789" 2024-05-08T05:53:43.151 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: trying mount with new device syntax: 0@708b9316-0cfd-11ef-bc97-c7b262605968.cephfs=/volumes/qa/sv_1/95b7cec1-e786-453a-8829-7f21d652d298 2024-05-08T05:53:43.151 INFO:teuthology.orchestra.run.smithi003.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.3:6789/172.21.15.89:6789/172.21.15.104:6789" will pass to kernel 2024-05-08T05:53:43.152 INFO:teuthology.orchestra.run.smithi003.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:53:43.152 INFO:teuthology.orchestra.run.smithi003.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:53:43.152 INFO:teuthology.orchestra.run.smithi003.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:53:43.152 INFO:teuthology.orchestra.run.smithi003.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:53:43.152 INFO:teuthology.orchestra.run.smithi003.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:53:43.154 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:53:43.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:53:43.154 DEBUG:teuthology.orchestra.run.smithi003:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:53:43.181 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:53:43.186 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:42.934+0000 7fb936aa7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:53:43.186 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:43.186+0000 7fb936aa7200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:53:43.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:53:43.240 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:43.246 INFO:teuthology.orchestra.run.smithi003.stdout:client15138 2024-05-08T05:53:43.246 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:53:43.246 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:53:43.264 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:53:43.357 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:53:43.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:43.365 INFO:teuthology.orchestra.run.smithi003.stdout:client15138 2024-05-08T05:53:43.365 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:53:43.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:53:43.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:43.425 INFO:teuthology.orchestra.run.smithi003.stdout:client15138 2024-05-08T05:53:43.425 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/708b9316-0cfd-11ef-bc97-c7b262605968.client15138/status') 2024-05-08T05:53:43.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:53:43.488 INFO:teuthology.orchestra.run.smithi003.stdout:instance: client.15138 (3)192.168.0.1:0/990261431 2024-05-08T05:53:43.488 INFO:teuthology.orchestra.run.smithi003.stdout:blocklisted: false 2024-05-08T05:53:43.488 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records in 2024-05-08T05:53:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:0+1 records out 2024-05-08T05:53:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:69 bytes copied, 2.457e-05 s, 2.8 MB/s 2024-05-08T05:53:43.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:53:43.496 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T05:53:43.507 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T05:53:43.637 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:53:43.638 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:53:43.638 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-08T05:53:43.639 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:43.179+0000 7fce6f838200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:53:43.639 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:43.276+0000 7fce6f838200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:53:43.639 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:43.167+0000 7f5db1f96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:53:43.639 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:43.261+0000 7f5db1f96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 -- 172.21.15.3:0/1319316597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 msgr2=0x7f9b70109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1319316597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b70109d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009ca0 tx=0x7f9b6802a8c0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 -- 172.21.15.3:0/1319316597 shutdown_connections 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1319316597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b70109d80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1319316597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b70102b30 0x7f9b70102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1319316597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 0x7f9b70101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.751 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 -- 172.21.15.3:0/1319316597 >> 172.21.15.3:0/1319316597 conn(0x7f9b700fced0 msgr2=0x7f9b700ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 -- 172.21.15.3:0/1319316597 shutdown_connections 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 -- 172.21.15.3:0/1319316597 wait complete. 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.749+0000 7f9b757c1640 1 Processor -- start 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 -- start start 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 0x7f9b70198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b70102b30 0x7f9b70199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b701a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010d470 con 0x7f9b701034f0 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010d2f0 con 0x7f9b70101940 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b757c1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7010d5f0 con 0x7f9b70102b30 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b701a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b701a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46028/0 (socket says 172.21.15.3:46028) 2024-05-08T05:53:43.752 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 -- 172.21.15.3:0/1707839145 learned_addr learned my addr 172.21.15.3:0/1707839145 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6e7fc640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b70102b30 0x7f9b70199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6effd640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 0x7f9b70198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 -- 172.21.15.3:0/1707839145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b70102b30 msgr2=0x7f9b70199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b70102b30 0x7f9b70199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 -- 172.21.15.3:0/1707839145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 msgr2=0x7f9b70198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 0x7f9b70198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6f7fe640 1 -- 172.21.15.3:0/1707839145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b701a0830 con 0x7f9b701034f0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.750+0000 7f9b6effd640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 0x7f9b70198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.751+0000 7f9b6f7fe640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b701a0190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b680097b0 tx=0x7f9b68031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.751+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b68003b70 con 0x7f9b701034f0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.751+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b68003d10 con 0x7f9b701034f0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.751+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b701a0ac0 con 0x7f9b701034f0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.751+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6802fba0 con 0x7f9b701034f0 2024-05-08T05:53:43.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.751+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b701a1010 con 0x7f9b701034f0 2024-05-08T05:53:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.752+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b701a11e0 con 0x7f9b701034f0 2024-05-08T05:53:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.753+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f9b6802fd40 con 0x7f9b701034f0 2024-05-08T05:53:43.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.754+0000 7f9b57fff640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 0x7f9b580b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:43.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.754+0000 7f9b6effd640 1 -- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 msgr2=0x7f9b580b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:43.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.754+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f9b68103c20 con 0x7f9b701034f0 2024-05-08T05:53:43.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.754+0000 7f9b6effd640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 0x7f9b580b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:53:43.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.759+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9b680cb550 con 0x7f9b701034f0 2024-05-08T05:53:43.767 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:43.658+0000 7fb936aa7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:53:43.954 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.954+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f9b701a13b0 con 0x7f9b701034f0 2024-05-08T05:53:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.955+0000 7f9b6effd640 1 -- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 msgr2=0x7f9b580b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:43.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.955+0000 7f9b6effd640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 0x7f9b580b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:53:43.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.957+0000 7f9b57fff640 1 -- 172.21.15.3:0/1707839145 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f9b680cae50 con 0x7f9b701034f0 2024-05-08T05:53:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 msgr2=0x7f9b580b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:53:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 0x7f9b580b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 msgr2=0x7f9b701a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b701a0190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b680097b0 tx=0x7f9b68031770 comp rx=0 tx=0).stop 2024-05-08T05:53:43.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 shutdown_connections 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f9b580b2830 0x7f9b580b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9b701034f0 0x7f9b701a0190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b70102b30 0x7f9b70199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 --2- 172.21.15.3:0/1707839145 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b70101940 0x7f9b70198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 >> 172.21.15.3:0/1707839145 conn(0x7f9b700fced0 msgr2=0x7f9b700fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 shutdown_connections 2024-05-08T05:53:43.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:43.960+0000 7f9b757c1640 1 -- 172.21.15.3:0/1707839145 wait complete. 2024-05-08T05:53:43.972 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-08T05:53:43.980 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:43.625+0000 7f5db1f96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:53:44.003 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:43.642+0000 7fce6f838200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:53:44.003 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.003+0000 7fce6f838200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 -- 172.21.15.3:0/1887424647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 msgr2=0x7fbd28101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 --2- 172.21.15.3:0/1887424647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28101d30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009900 tx=0x7fbd1802a410 comp rx=0 tx=0).stop 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 -- 172.21.15.3:0/1887424647 shutdown_connections 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 --2- 172.21.15.3:0/1887424647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd281034c0 0x7fbd28109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 --2- 172.21.15.3:0/1887424647 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 0x7fbd28102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 --2- 172.21.15.3:0/1887424647 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28101d30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 -- 172.21.15.3:0/1887424647 >> 172.21.15.3:0/1887424647 conn(0x7fbd280fce80 msgr2=0x7fbd280ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 -- 172.21.15.3:0/1887424647 shutdown_connections 2024-05-08T05:53:44.081 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.078+0000 7fbd2e474640 1 -- 172.21.15.3:0/1887424647 wait complete. 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 Processor -- start 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 -- start start 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 0x7fbd28199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd281034c0 0x7fbd281a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810d460 con 0x7fbd28101910 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810d2e0 con 0x7fbd28102b00 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd2e474640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2810d5e0 con 0x7fbd281034c0 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd27fff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46058/0 (socket says 172.21.15.3:46058) 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd27fff640 1 -- 172.21.15.3:0/8742869 learned_addr learned my addr 172.21.15.3:0/8742869 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.079+0000 7fbd277fe640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 0x7fbd28199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.082 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd2ca11640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd281034c0 0x7fbd281a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd277fe640 1 -- 172.21.15.3:0/8742869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd281034c0 msgr2=0x7fbd281a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd277fe640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd281034c0 0x7fbd281a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd277fe640 1 -- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 msgr2=0x7fbd28198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd277fe640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd277fe640 1 -- 172.21.15.3:0/8742869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd281a0920 con 0x7fbd28102b00 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd27fff640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd277fe640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 0x7fbd28199250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000e9f0 tx=0x7fbd1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000ccc0 con 0x7fbd28102b00 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd281a0c10 con 0x7fbd28102b00 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd100108e0 con 0x7fbd28102b00 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10018880 con 0x7fbd28102b00 2024-05-08T05:53:44.083 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.080+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd281a11b0 con 0x7fbd28102b00 2024-05-08T05:53:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.081+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd281a1380 con 0x7fbd28102b00 2024-05-08T05:53:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.083+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7fbd10010430 con 0x7fbd28102b00 2024-05-08T05:53:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.083+0000 7fbd257fa640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 0x7fbd140b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.083+0000 7fbd27fff640 1 -- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 msgr2=0x7fbd140b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:44.084 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.083+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7fbd100e29d0 con 0x7fbd28102b00 2024-05-08T05:53:44.085 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.083+0000 7fbd27fff640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 0x7fbd140b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:53:44.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.087+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd100a9d10 con 0x7fbd28102b00 2024-05-08T05:53:44.159 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:43.767+0000 7fb936aa7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:53:44.262 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:43.980+0000 7f5db1f96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:53:44.262 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:44.071+0000 7f5db1f96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:53:44.270 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.096+0000 7fce6f838200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:53:44.280 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.280+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fbd281a1550 con 0x7fbd28102b00 2024-05-08T05:53:44.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.284+0000 7fbd27fff640 1 -- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 msgr2=0x7fbd140b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:44.284 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.284+0000 7fbd27fff640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 0x7fbd140b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:53:44.286 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.286+0000 7fbd257fa640 1 -- 172.21.15.3:0/8742869 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fbd100b23e0 con 0x7fbd28102b00 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 msgr2=0x7fbd140b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 0x7fbd140b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 msgr2=0x7fbd28199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 0x7fbd28199250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000e9f0 tx=0x7fbd1000eeb0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 shutdown_connections 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7fbd140b2830 0x7fbd140b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd281034c0 0x7fbd281a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd28102b00 0x7fbd28199250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 --2- 172.21.15.3:0/8742869 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbd28101910 0x7fbd28198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.289 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 >> 172.21.15.3:0/8742869 conn(0x7fbd280fce80 msgr2=0x7fbd2806a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.288+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 shutdown_connections 2024-05-08T05:53:44.290 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.289+0000 7fbd2e474640 1 -- 172.21.15.3:0/8742869 wait complete. 2024-05-08T05:53:44.300 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 -- 172.21.15.3:0/625096383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c1086f0 msgr2=0x7f205c10ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 --2- 172.21.15.3:0/625096383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c1086f0 0x7f205c10ab40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2054009ca0 tx=0x7f205402a8c0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 -- 172.21.15.3:0/625096383 shutdown_connections 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 --2- 172.21.15.3:0/625096383 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c1086f0 0x7f205c10ab40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 --2- 172.21.15.3:0/625096383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205c105dc0 0x7f205c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 --2- 172.21.15.3:0/625096383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f205c103490 0x7f205c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 -- 172.21.15.3:0/625096383 >> 172.21.15.3:0/625096383 conn(0x7f205c0fce80 msgr2=0x7f205c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 -- 172.21.15.3:0/625096383 shutdown_connections 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.406+0000 7f20608e1640 1 -- 172.21.15.3:0/625096383 wait complete. 2024-05-08T05:53:44.409 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 Processor -- start 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 -- start start 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f205c103490 0x7f205c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 0x7f205c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205c1086f0 0x7f205c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10e560 con 0x7f205c105dc0 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10e3e0 con 0x7f205c1086f0 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.407+0000 7f20608e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f205c10e6e0 con 0x7f205c103490 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 0x7f205c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 0x7f205c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46082/0 (socket says 172.21.15.3:46082) 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 -- 172.21.15.3:0/646801418 learned_addr learned my addr 172.21.15.3:0/646801418 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f205a59c640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f205c103490 0x7f205c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f205ad9d640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205c1086f0 0x7f205c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 -- 172.21.15.3:0/646801418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f205c103490 msgr2=0x7f205c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.410 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f205c103490 0x7f205c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 -- 172.21.15.3:0/646801418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205c1086f0 msgr2=0x7f205c1a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205c1086f0 0x7f205c1a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 -- 172.21.15.3:0/646801418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205c1a0840 con 0x7f205c105dc0 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f2059d9b640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 0x7f205c199170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f205000e9c0 tx=0x7f205000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205000cd80 con 0x7f205c105dc0 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20500054d0 con 0x7f205c105dc0 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205c1a0b30 con 0x7f205c105dc0 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050010430 con 0x7f205c105dc0 2024-05-08T05:53:44.411 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.408+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205c1a1080 con 0x7f205c105dc0 2024-05-08T05:53:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.409+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f205c0fe970 con 0x7f205c105dc0 2024-05-08T05:53:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.411+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7f2050004fb0 con 0x7f205c105dc0 2024-05-08T05:53:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.411+0000 7f204b7fe640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 0x7f20440b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.412+0000 7f205a59c640 1 -- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 msgr2=0x7f20440b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.412+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7f20500e2cb0 con 0x7f205c105dc0 2024-05-08T05:53:44.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.412+0000 7f205a59c640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 0x7f20440b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:53:44.415 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.415+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2050054df0 con 0x7f205c105dc0 2024-05-08T05:53:44.556 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:44.166+0000 7fb936aa7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:53:44.556 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:44.555+0000 7fb936aa7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:53:44.608 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.608+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f205c02db50 con 0x7f205c105dc0 2024-05-08T05:53:44.611 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.612+0000 7f204b7fe640 1 -- 172.21.15.3:0/646801418 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f20500aa640 con 0x7f205c105dc0 2024-05-08T05:53:44.612 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.613+0000 7f205a59c640 1 -- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 msgr2=0x7f20440b4cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:44.613 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.613+0000 7f205a59c640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 0x7f20440b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:53:44.614 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 msgr2=0x7f20440b4cf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 0x7f20440b4cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 msgr2=0x7f205c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 0x7f205c199170 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f205000e9c0 tx=0x7f205000ee80 comp rx=0 tx=0).stop 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 shutdown_connections 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7f20440b2830 0x7f20440b4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f205c1086f0 0x7f205c1a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f205c105dc0 0x7f205c199170 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 --2- 172.21.15.3:0/646801418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f205c103490 0x7f205c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 >> 172.21.15.3:0/646801418 conn(0x7f205c0fce80 msgr2=0x7f205c103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 shutdown_connections 2024-05-08T05:53:44.615 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.614+0000 7f20608e1640 1 -- 172.21.15.3:0/646801418 wait complete. 2024-05-08T05:53:44.626 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-08T05:53:44.736 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 -- 172.21.15.3:0/3980540576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 msgr2=0x7ffa5c10ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 --2- 172.21.15.3:0/3980540576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 0x7ffa5c10ae70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009ca0 tx=0x7ffa5402a8c0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 -- 172.21.15.3:0/3980540576 shutdown_connections 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 --2- 172.21.15.3:0/3980540576 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 0x7ffa5c10ae70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 --2- 172.21.15.3:0/3980540576 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa5c103c20 0x7ffa5c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 --2- 172.21.15.3:0/3980540576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa5c102a30 0x7ffa5c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 -- 172.21.15.3:0/3980540576 >> 172.21.15.3:0/3980540576 conn(0x7ffa5c0fdfa0 msgr2=0x7ffa5c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 -- 172.21.15.3:0/3980540576 shutdown_connections 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.734+0000 7ffa61446640 1 -- 172.21.15.3:0/3980540576 wait complete. 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 Processor -- start 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 -- start start 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa5c102a30 0x7ffa5c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.737 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa5c103c20 0x7ffa5c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 0x7ffa5c1a6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c114be0 con 0x7ffa5c1045e0 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c114a60 con 0x7ffa5c102a30 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa61446640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c114d60 con 0x7ffa5c103c20 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa5b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 0x7ffa5c1a6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa5c103c20 0x7ffa5c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa5b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 0x7ffa5c1a6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46096/0 (socket says 172.21.15.3:46096) 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa5b7fe640 1 -- 172.21.15.3:0/3399414425 learned_addr learned my addr 172.21.15.3:0/3399414425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.735+0000 7ffa5affd640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa5c102a30 0x7ffa5c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa5affd640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa5c103c20 msgr2=0x7ffa5c19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa5affd640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa5c103c20 0x7ffa5c19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa5affd640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 msgr2=0x7ffa5c1a6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:44.738 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa5affd640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ffa5c1045e0 0x7ffa5c1a6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa5affd640 1 -- 172.21.15.3:0/3399414425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1a6de0 con 0x7ffa5c102a30 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa5affd640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa5c102a30 0x7ffa5c19f210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9c0 tx=0x7ffa4c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00ccc0 con 0x7ffa5c102a30 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4c0035f0 con 0x7ffa5c102a30 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1a70d0 con 0x7ffa5c102a30 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00ce60 con 0x7ffa5c102a30 2024-05-08T05:53:44.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.736+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1a75d0 con 0x7ffa5c102a30 2024-05-08T05:53:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.737+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c109da0 con 0x7ffa5c102a30 2024-05-08T05:53:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.739+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 19) ==== 149668+0+0 (secure 0 0 0) 0x7ffa4c004000 con 0x7ffa5c102a30 2024-05-08T05:53:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.739+0000 7ffa58ff9640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 0x7ffa480b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.739+0000 7ffa53fff640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 msgr2=0x7ffa480b4ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.739+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9150+0+0 (secure 0 0 0) 0x7ffa4c0e2970 con 0x7ffa5c102a30 2024-05-08T05:53:44.740 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.739+0000 7ffa53fff640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 0x7ffa480b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:53:44.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.742+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ffa4c0aa2a0 con 0x7ffa5c102a30 2024-05-08T05:53:44.762 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:44.312+0000 7f5db1f96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:53:44.762 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:44.443+0000 7f5db1f96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:53:44.762 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:44.538+0000 7f5db1f96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:53:44.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.339+0000 7fce6f838200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:53:44.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.474+0000 7fce6f838200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:53:44.771 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.571+0000 7fce6f838200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:53:44.909 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:44.657+0000 7fb936aa7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:53:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.940+0000 7ffa53fff640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 msgr2=0x7ffa480b4ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:44.940 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.940+0000 7ffa53fff640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 0x7ffa480b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:53:44.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:44.960+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 --> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ffa5c062df0 con 0x7ffa480b2820 2024-05-08T05:53:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:44 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1707839145' entity='client.admin' 2024-05-08T05:53:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:44 smithi003 ceph-mon[30137]: from='client.? ' entity='client.admin' 2024-05-08T05:53:45.177 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:44 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/646801418' entity='client.admin' 2024-05-08T05:53:45.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:44.925+0000 7fb936aa7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:53:45.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:45.072+0000 7fb936aa7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:53:45.177 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:45.176+0000 7fb936aa7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:53:45.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:44 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1707839145' entity='client.admin' 2024-05-08T05:53:45.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:44 smithi089 ceph-mon[34495]: from='client.? ' entity='client.admin' 2024-05-08T05:53:45.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:44 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/646801418' entity='client.admin' 2024-05-08T05:53:45.263 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:44.861+0000 7f5db1f96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:53:45.263 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:44.951+0000 7f5db1f96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:53:45.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:44 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1707839145' entity='client.admin' 2024-05-08T05:53:45.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:44 smithi104 ceph-mon[34559]: from='client.? ' entity='client.admin' 2024-05-08T05:53:45.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:44 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/646801418' entity='client.admin' 2024-05-08T05:53:45.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.894+0000 7fce6f838200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:53:45.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:44.986+0000 7fce6f838200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:53:45.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:45.341+0000 7ffa53fff640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 msgr2=0x7ffa480b4ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:45.341 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:45.341+0000 7ffa53fff640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 0x7ffa480b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:53:45.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:45.520+0000 7fb936aa7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:53:45.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:45.620+0000 7fb936aa7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:53:45.958 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:45.582+0000 7fce6f838200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:53:45.958 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 05:53:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z[36345]: 2024-05-08T05:53:45.795+0000 7fce6f838200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:53:45.959 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:45 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:45.545+0000 7f5db1f96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:53:45.959 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 05:53:45 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y[35917]: 2024-05-08T05:53:45.757+0000 7f5db1f96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:53:45.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:45.974+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mgrmap(e 20) ==== 149668+0+0 (secure 0 0 0) 0x7ffa4c0a5e30 con 0x7ffa5c102a30 2024-05-08T05:53:46.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:46.142+0000 7ffa53fff640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 msgr2=0x7ffa480b4ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.3:6800/2793699832 2024-05-08T05:53:46.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:46.142+0000 7ffa53fff640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 0x7ffa480b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:53:46.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: Standby manager daemon y restarted 2024-05-08T05:53:46.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: Standby manager daemon y started 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: Standby manager daemon z restarted 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: Standby manager daemon z started 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:53:46.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:45 smithi089 ceph-mon[34495]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:53:46.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: Standby manager daemon y restarted 2024-05-08T05:53:46.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: Standby manager daemon y started 2024-05-08T05:53:46.269 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: Standby manager daemon z restarted 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: Standby manager daemon z started 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:53:46.270 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:45 smithi003 ceph-mon[30137]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: Standby manager daemon y restarted 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: Standby manager daemon y started 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.89:0/2288433712' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: Standby manager daemon z restarted 2024-05-08T05:53:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: Standby manager daemon z started 2024-05-08T05:53:46.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:53:46.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:53:46.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:53:46.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:45 smithi104 ceph-mon[34559]: from='mgr.? 172.21.15.104:0/3912520461' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:53:46.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:46.518+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7ffa4c0e0100 con 0x7ffa5c102a30 2024-05-08T05:53:46.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:46.518+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 msgr2=0x7ffa480b4ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:53:46.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:46.518+0000 7ffa58ff9640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/2793699832,v1:172.21.15.3:6801/2793699832] conn(0x7ffa480b2820 0x7ffa480b4ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:46.654 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.269+0000 7fb936aa7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:53:46.654 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.501+0000 7fb936aa7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb893b34700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb893b34700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb893b34700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb893b34700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb893b34700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb89733b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb89733b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.910 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb89733b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.911 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb89733b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:46.911 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:53:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:53:46.753+0000 7fb89733b700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: Active manager daemon x restarted 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: Activating manager daemon x 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: mgrmap e21: x(active, starting, since 0.0127631s), standbys: y, z 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:53:47.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:53:47.264 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:53:47.265 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: Manager daemon x is now available 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:53:47.266 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:46 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: Active manager daemon x restarted 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: Activating manager daemon x 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: mgrmap e21: x(active, starting, since 0.0127631s), standbys: y, z 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:53:47.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:53:47.272 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:53:47.273 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: Manager daemon x is now available 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:53:47.274 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:46 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:53:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-05-08T05:53:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: Active manager daemon x restarted 2024-05-08T05:53:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: Activating manager daemon x 2024-05-08T05:53:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T05:53:47.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: mgrmap e21: x(active, starting, since 0.0127631s), standbys: y, z 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:53:47.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:53:47.412 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:53:47.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:53:47.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: Manager daemon x is now available 2024-05-08T05:53:47.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:47.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:53:47.413 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:46 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:53:47.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.523+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mon.2 v2:172.21.15.89:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7ffa4c00ccc0 con 0x7ffa5c102a30 2024-05-08T05:53:47.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.524+0000 7ffa58ff9640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ffa4803a8a0 0x7ffa4803ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:47.524 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.524+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ffa5c062df0 con 0x7ffa4803a8a0 2024-05-08T05:53:47.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.527+0000 7ffa53fff640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ffa4803a8a0 0x7ffa4803ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:47.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.529+0000 7ffa53fff640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ffa4803a8a0 0x7ffa4803ac80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c1a06d0 tx=0x7ffa44005f90 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:47.596 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.596+0000 7ffa58ff9640 1 -- 172.21.15.3:0/3399414425 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffa5c062df0 con 0x7ffa4803a8a0 2024-05-08T05:53:47.596 INFO:teuthology.orchestra.run.smithi003.stdout:Schedule set for path / 2024-05-08T05:53:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ffa4803a8a0 msgr2=0x7ffa4803ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ffa4803a8a0 0x7ffa4803ac80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c1a06d0 tx=0x7ffa44005f90 comp rx=0 tx=0).stop 2024-05-08T05:53:47.598 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa5c102a30 msgr2=0x7ffa5c19f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 --2- 172.21.15.3:0/3399414425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ffa5c102a30 0x7ffa5c19f210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00e9c0 tx=0x7ffa4c00ee80 comp rx=0 tx=0).stop 2024-05-08T05:53:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa5b7fe640 1 -- 172.21.15.3:0/3399414425 reap_dead start 2024-05-08T05:53:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 shutdown_connections 2024-05-08T05:53:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 >> 172.21.15.3:0/3399414425 conn(0x7ffa5c0fdfa0 msgr2=0x7ffa5c106f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 shutdown_connections 2024-05-08T05:53:47.599 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.598+0000 7ffa61446640 1 -- 172.21.15.3:0/3399414425 wait complete. 2024-05-08T05:53:47.609 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-08T05:53:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 -- 172.21.15.3:0/3351354945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc1067f0 msgr2=0x7f78bc10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 --2- 172.21.15.3:0/3351354945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc1067f0 0x7f78bc10d080 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009900 tx=0x7f78b402a410 comp rx=0 tx=0).stop 2024-05-08T05:53:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 -- 172.21.15.3:0/3351354945 shutdown_connections 2024-05-08T05:53:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 --2- 172.21.15.3:0/3351354945 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc1067f0 0x7f78bc10d080 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:47.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 --2- 172.21.15.3:0/3351354945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc105e30 0x7f78bc1062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 --2- 172.21.15.3:0/3351354945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78bc104c40 0x7f78bc105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 -- 172.21.15.3:0/3351354945 >> 172.21.15.3:0/3351354945 conn(0x7f78bc1001d0 msgr2=0x7f78bc1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 -- 172.21.15.3:0/3351354945 shutdown_connections 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.720+0000 7f78c315f640 1 -- 172.21.15.3:0/3351354945 wait complete. 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 Processor -- start 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 -- start start 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc104c40 0x7f78bc19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 0x7f78bc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78bc1067f0 0x7f78bc1a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc110750 con 0x7f78bc105e30 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1105d0 con 0x7f78bc104c40 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c315f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1108d0 con 0x7f78bc1067f0 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 0x7f78bc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:47.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 0x7f78bc19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46206/0 (socket says 172.21.15.3:46206) 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 -- 172.21.15.3:0/3987303641 learned_addr learned my addr 172.21.15.3:0/3987303641 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c0efb640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc104c40 0x7f78bc19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c16fc640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78bc1067f0 0x7f78bc1a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 -- 172.21.15.3:0/3987303641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78bc1067f0 msgr2=0x7f78bc1a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78bc1067f0 0x7f78bc1a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 -- 172.21.15.3:0/3987303641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc104c40 msgr2=0x7f78bc19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc104c40 0x7f78bc19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78bbfff640 1 -- 172.21.15.3:0/3987303641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc1a2a50 con 0x7f78bc105e30 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.721+0000 7f78c0efb640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc104c40 0x7f78bc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.722+0000 7f78bbfff640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 0x7f78bc19b370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ebc0 tx=0x7f78a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.722+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4010630 con 0x7f78bc105e30 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.722+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc1a2d40 con 0x7f78bc105e30 2024-05-08T05:53:47.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.722+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a4010c80 con 0x7f78bc105e30 2024-05-08T05:53:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.722+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4018b60 con 0x7f78bc105e30 2024-05-08T05:53:47.725 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.722+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a3230 con 0x7f78bc105e30 2024-05-08T05:53:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.723+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78bc10c0d0 con 0x7f78bc105e30 2024-05-08T05:53:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.724+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f78a4018d00 con 0x7f78bc105e30 2024-05-08T05:53:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.725+0000 7f78b9ffb640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78a80b25e0 0x7f78a80b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.725+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f78a40e2880 con 0x7f78bc105e30 2024-05-08T05:53:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.725+0000 7f78c0efb640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78a80b25e0 0x7f78a80b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:47.726 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.726+0000 7f78c0efb640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78a80b25e0 0x7f78a80b4aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78bc105c90 tx=0x7f78ac005f90 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:47.729 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.730+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78a40a9d90 con 0x7f78bc105e30 2024-05-08T05:53:47.946 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:47.946+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f78bc062f50 con 0x7f78a80b25e0 2024-05-08T05:53:48.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.008+0000 7f78b9ffb640 1 -- 172.21.15.3:0/3987303641 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f78bc062f50 con 0x7f78a80b25e0 2024-05-08T05:53:48.008 INFO:teuthology.orchestra.run.smithi003.stdout:Retention added to path / 2024-05-08T05:53:48.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78a80b25e0 msgr2=0x7f78a80b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78a80b25e0 0x7f78a80b4aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78bc105c90 tx=0x7f78ac005f90 comp rx=0 tx=0).stop 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 msgr2=0x7f78bc19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 0x7f78bc19b370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f78a400ebc0 tx=0x7f78a400c6a0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 shutdown_connections 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78a80b25e0 0x7f78a80b4aa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78bc1067f0 0x7f78bc1a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f78bc105e30 0x7f78bc19b370 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 --2- 172.21.15.3:0/3987303641 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f78bc104c40 0x7f78bc19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 >> 172.21.15.3:0/3987303641 conn(0x7f78bc1001d0 msgr2=0x7f78bc1092a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 shutdown_connections 2024-05-08T05:53:48.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.010+0000 7f78c315f640 1 -- 172.21.15.3:0/3987303641 wait complete. 2024-05-08T05:53:48.022 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T05:53:48.139 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 -- 172.21.15.3:0/3892766773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 msgr2=0x7f1dc4109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 --2- 172.21.15.3:0/3892766773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 0x7f1dc4109d50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009ca0 tx=0x7f1dbc02a8c0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 -- 172.21.15.3:0/3892766773 shutdown_connections 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 --2- 172.21.15.3:0/3892766773 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 0x7f1dc4109d50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 --2- 172.21.15.3:0/3892766773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4102b00 0x7f1dc4102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 --2- 172.21.15.3:0/3892766773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dc4101910 0x7f1dc4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 -- 172.21.15.3:0/3892766773 >> 172.21.15.3:0/3892766773 conn(0x7f1dc40fce80 msgr2=0x7f1dc40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 -- 172.21.15.3:0/3892766773 shutdown_connections 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 -- 172.21.15.3:0/3892766773 wait complete. 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.137+0000 7f1dca140640 1 Processor -- start 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 -- start start 2024-05-08T05:53:48.140 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 0x7f1dc4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dc4102b00 0x7f1dc4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 0x7f1dc41a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc410d440 con 0x7f1dc41034c0 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc410d2c0 con 0x7f1dc4101910 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dca140640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc410d5c0 con 0x7f1dc4102b00 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 0x7f1dc4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 0x7f1dc4198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:53812/0 (socket says 172.21.15.3:53812) 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 -- 172.21.15.3:0/1461695990 learned_addr learned my addr 172.21.15.3:0/1461695990 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc3fff640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 0x7f1dc41a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc2ffd640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dc4102b00 0x7f1dc4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 -- 172.21.15.3:0/1461695990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dc4102b00 msgr2=0x7f1dc4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dc4102b00 0x7f1dc4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.141 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 -- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 msgr2=0x7f1dc41a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 0x7f1dc41a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.138+0000 7f1dc37fe640 1 -- 172.21.15.3:0/1461695990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41a0860 con 0x7f1dc4101910 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.139+0000 7f1dc37fe640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 0x7f1dc4198d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d950 tx=0x7f1db400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.139+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db4017070 con 0x7f1dc4101910 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.139+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1db4009c30 con 0x7f1dc4101910 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.139+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a0b50 con 0x7f1dc4101910 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.139+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db400f620 con 0x7f1dc4101910 2024-05-08T05:53:48.142 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.139+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a1050 con 0x7f1dc4101910 2024-05-08T05:53:48.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.140+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc41a1220 con 0x7f1dc4101910 2024-05-08T05:53:48.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.141+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f1db401a070 con 0x7f1dc4101910 2024-05-08T05:53:48.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.142+0000 7f1dc0ff9640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1db00b25e0 0x7f1db00b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.143 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.142+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f1db40e1d00 con 0x7f1dc4101910 2024-05-08T05:53:48.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.142+0000 7f1dc2ffd640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1db00b25e0 0x7f1db00b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.144 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.143+0000 7f1dc2ffd640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1db00b25e0 0x7f1db00b4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009bd0 tx=0x7f1dac005f90 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:48.145 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.145+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1db4009de0 con 0x7f1dc4101910 2024-05-08T05:53:48.362 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.361+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f1dc41a13f0 con 0x7f1db00b25e0 2024-05-08T05:53:48.369 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.369+0000 7f1dc0ff9640 1 -- 172.21.15.3:0/1461695990 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f1dc41a13f0 con 0x7f1db00b25e0 2024-05-08T05:53:48.369 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T05:53:47", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1db00b25e0 msgr2=0x7f1db00b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1db00b25e0 0x7f1db00b4aa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1dac009bd0 tx=0x7f1dac005f90 comp rx=0 tx=0).stop 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 msgr2=0x7f1dc4198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 0x7f1dc4198d10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1db400d950 tx=0x7f1db400de10 comp rx=0 tx=0).stop 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 shutdown_connections 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1db00b25e0 0x7f1db00b4aa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1dc41034c0 0x7f1dc41a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1dc4102b00 0x7f1dc4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.372 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 --2- 172.21.15.3:0/1461695990 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1dc4101910 0x7f1dc4198d10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 >> 172.21.15.3:0/1461695990 conn(0x7f1dc40fce80 msgr2=0x7f1dc40fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 shutdown_connections 2024-05-08T05:53:48.373 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.371+0000 7f1dca140640 1 -- 172.21.15.3:0/1461695990 wait complete. 2024-05-08T05:53:48.384 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 -- 172.21.15.3:0/4061396061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 msgr2=0x7fd3d4105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 --2- 172.21.15.3:0/4061396061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4105880 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8009900 tx=0x7fd3c802a410 comp rx=0 tx=0).stop 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 -- 172.21.15.3:0/4061396061 shutdown_connections 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 --2- 172.21.15.3:0/4061396061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d41086f0 0x7fd3d410ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 --2- 172.21.15.3:0/4061396061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d4105dc0 0x7fd3d41081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 --2- 172.21.15.3:0/4061396061 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4105880 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 -- 172.21.15.3:0/4061396061 >> 172.21.15.3:0/4061396061 conn(0x7fd3d40fce80 msgr2=0x7fd3d40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 -- 172.21.15.3:0/4061396061 shutdown_connections 2024-05-08T05:53:48.499 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.498+0000 7fd3db132640 1 -- 172.21.15.3:0/4061396061 wait complete. 2024-05-08T05:53:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 Processor -- start 2024-05-08T05:53:48.501 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 -- start start 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d4105dc0 0x7fd3d4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d41086f0 0x7fd3d41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410e570 con 0x7fd3d4103490 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410e3f0 con 0x7fd3d4105dc0 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.499+0000 7fd3db132640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3d410e6f0 con 0x7fd3d41086f0 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d96cf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d41086f0 0x7fd3d41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46264/0 (socket says 172.21.15.3:46264) 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 -- 172.21.15.3:0/3698682351 learned_addr learned my addr 172.21.15.3:0/3698682351 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d3fff640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d4105dc0 0x7fd3d4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 -- 172.21.15.3:0/3698682351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d41086f0 msgr2=0x7fd3d41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.502 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d41086f0 0x7fd3d41a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 -- 172.21.15.3:0/3698682351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d4105dc0 msgr2=0x7fd3d4199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d4105dc0 0x7fd3d4199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 -- 172.21.15.3:0/3698682351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3d41a0900 con 0x7fd3d4103490 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d3fff640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d4105dc0 0x7fd3d4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d8ece640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4198cf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005940 tx=0x7fd3c8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c803f030 con 0x7fd3d4103490 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3d41a0b90 con 0x7fd3d4103490 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3c8003aa0 con 0x7fd3d4103490 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.500+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8035070 con 0x7fd3d4103490 2024-05-08T05:53:48.503 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.501+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3d41a10d0 con 0x7fd3d4103490 2024-05-08T05:53:48.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.502+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3d40fe970 con 0x7fd3d4103490 2024-05-08T05:53:48.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.503+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fd3c8042030 con 0x7fd3d4103490 2024-05-08T05:53:48.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.504+0000 7fd3d1ffb640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd3c40b25e0 0x7fd3c40b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:48.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.505+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7fd3c8102890 con 0x7fd3d4103490 2024-05-08T05:53:48.504 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.505+0000 7fd3d3fff640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd3c40b25e0 0x7fd3c40b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:48.505 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.505+0000 7fd3d3fff640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd3c40b25e0 0x7fd3c40b4aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0005fd0 tx=0x7fd3c000b390 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:48.513 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.513+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3c80c9dc0 con 0x7fd3d4103490 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: [08/May/2024:05:53:47] ENGINE Bus STARTING 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: [08/May/2024:05:53:47] ENGINE Serving on http://172.21.15.3:8765 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: [08/May/2024:05:53:47] ENGINE Serving on https://172.21.15.3:7150 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: [08/May/2024:05:53:47] ENGINE Bus STARTED 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: mgrmap e22: x(active, since 1.02107s), standbys: y, z 2024-05-08T05:53:48.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: from='client.24920 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:48.661 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:48 smithi003 ceph-mon[30137]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:48.732 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.732+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd3d4062df0 con 0x7fd3c40b25e0 2024-05-08T05:53:48.739 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.739+0000 7fd3d1ffb640 1 -- 172.21.15.3:0/3698682351 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd3d4062df0 con 0x7fd3c40b25e0 2024-05-08T05:53:48.739 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-05-08T05:53:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd3c40b25e0 msgr2=0x7fd3c40b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd3c40b25e0 0x7fd3c40b4aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0005fd0 tx=0x7fd3c000b390 comp rx=0 tx=0).stop 2024-05-08T05:53:48.741 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 msgr2=0x7fd3d4198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4198cf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8005940 tx=0x7fd3c8031a50 comp rx=0 tx=0).stop 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 shutdown_connections 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd3c40b25e0 0x7fd3c40b4aa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd3d41086f0 0x7fd3d41a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd3d4105dc0 0x7fd3d4199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 --2- 172.21.15.3:0/3698682351 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd3d4103490 0x7fd3d4198cf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 >> 172.21.15.3:0/3698682351 conn(0x7fd3d40fce80 msgr2=0x7fd3d4105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 shutdown_connections 2024-05-08T05:53:48.742 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:48.741+0000 7fd3db132640 1 -- 172.21.15.3:0/3698682351 wait complete. 2024-05-08T05:53:48.752 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-08T05:53:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: [08/May/2024:05:53:47] ENGINE Bus STARTING 2024-05-08T05:53:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: [08/May/2024:05:53:47] ENGINE Serving on http://172.21.15.3:8765 2024-05-08T05:53:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: [08/May/2024:05:53:47] ENGINE Serving on https://172.21.15.3:7150 2024-05-08T05:53:48.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: [08/May/2024:05:53:47] ENGINE Bus STARTED 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: mgrmap e22: x(active, since 1.02107s), standbys: y, z 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: from='client.24920 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:48.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:48 smithi089 ceph-mon[34495]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: [08/May/2024:05:53:47] ENGINE Bus STARTING 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: [08/May/2024:05:53:47] ENGINE Serving on http://172.21.15.3:8765 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: [08/May/2024:05:53:47] ENGINE Serving on https://172.21.15.3:7150 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: [08/May/2024:05:53:47] ENGINE Bus STARTED 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: mgrmap e22: x(active, since 1.02107s), standbys: y, z 2024-05-08T05:53:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: from='client.24920 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:48.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:48 smithi104 ceph-mon[34559]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:48.780 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-08T05:53:48.792 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-08T05:53:48.801 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:53:48.801 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-08T05:53:48.823 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-08T05:53:48.836 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:53:48.837 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:53:49.124 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:49.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:49.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:49 smithi003 ceph-mon[30137]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T05:53:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:49 smithi089 ceph-mon[34495]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='client.15204 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:49 smithi104 ceph-mon[34559]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 -- 172.21.15.3:0/1764581838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190109520 msgr2=0x7fe190109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 --2- 172.21.15.3:0/1764581838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190109520 0x7fe190109900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe180009990 tx=0x7fe180009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 -- 172.21.15.3:0/1764581838 shutdown_connections 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 --2- 172.21.15.3:0/1764581838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe190102450 0x7fe190110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 --2- 172.21.15.3:0/1764581838 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190109520 0x7fe190109900 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 --2- 172.21.15.3:0/1764581838 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190101a90 0x7fe190101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 -- 172.21.15.3:0/1764581838 >> 172.21.15.3:0/1764581838 conn(0x7fe190078a10 msgr2=0x7fe1900ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:50.866 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.865+0000 7fe196a28700 1 -- 172.21.15.3:0/1764581838 shutdown_connections 2024-05-08T05:53:50.867 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.866+0000 7fe196a28700 1 -- 172.21.15.3:0/1764581838 wait complete. 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 Processor -- start 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 -- start start 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 0x7fe19019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe190102450 0x7fe19019fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190109520 0x7fe1901999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe190113650 con 0x7fe190101a90 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901134d0 con 0x7fe190102450 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe196a28700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe1901137d0 con 0x7fe190109520 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe18ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 0x7fe19019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:50.868 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe18ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 0x7fe19019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46320/0 (socket says 172.21.15.3:46320) 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.867+0000 7fe18ffff700 1 -- 172.21.15.3:0/4144289937 learned_addr learned my addr 172.21.15.3:0/4144289937 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe194fc5700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190109520 0x7fe1901999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18f7fe700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe190102450 0x7fe19019fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18ffff700 1 -- 172.21.15.3:0/4144289937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190109520 msgr2=0x7fe1901999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18ffff700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190109520 0x7fe1901999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18ffff700 1 -- 172.21.15.3:0/4144289937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe190102450 msgr2=0x7fe19019fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18ffff700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe190102450 0x7fe19019fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18ffff700 1 -- 172.21.15.3:0/4144289937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19019a150 con 0x7fe190101a90 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe194fc5700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190109520 0x7fe1901999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:50.869 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18ffff700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 0x7fe19019f830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe18000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180027070 con 0x7fe190101a90 2024-05-08T05:53:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe180022470 con 0x7fe190101a90 2024-05-08T05:53:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19019a3e0 con 0x7fe190101a90 2024-05-08T05:53:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180017650 con 0x7fe190101a90 2024-05-08T05:53:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.868+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1901a6700 con 0x7fe190101a90 2024-05-08T05:53:50.870 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.869+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19010ba00 con 0x7fe190101a90 2024-05-08T05:53:50.872 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.872+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 23) ==== 149771+0+0 (secure 0 0 0) 0x7fe18002a070 con 0x7fe190101a90 2024-05-08T05:53:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.873+0000 7fe18d7fa700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe1780b29f0 0x7fe1780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.873+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7fe1800e3830 con 0x7fe190101a90 2024-05-08T05:53:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.873+0000 7fe18f7fe700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe1780b29f0 0x7fe1780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:50.873 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.873+0000 7fe18f7fe700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe1780b29f0 0x7fe1780b4eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe1900ffa40 tx=0x7fe184009260 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:50.875 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:50.875+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe180013070 con 0x7fe190101a90 2024-05-08T05:53:51.087 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.086+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe19019af20 con 0x7fe1780b29f0 2024-05-08T05:53:51.097 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.097+0000 7fe18d7fa700 1 -- 172.21.15.3:0/4144289937 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339575 (secure 0 0 0) 0x7fe19019af20 con 0x7fe1780b29f0 2024-05-08T05:53:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe1780b29f0 msgr2=0x7fe1780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe1780b29f0 0x7fe1780b4eb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe1900ffa40 tx=0x7fe184009260 comp rx=0 tx=0).stop 2024-05-08T05:53:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 msgr2=0x7fe19019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 0x7fe19019f830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe180000c00 tx=0x7fe18000bfd0 comp rx=0 tx=0).stop 2024-05-08T05:53:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 shutdown_connections 2024-05-08T05:53:51.105 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe1780b29f0 0x7fe1780b4eb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe190109520 0x7fe1901999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe190101a90 0x7fe19019f830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 --2- 172.21.15.3:0/4144289937 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe190102450 0x7fe19019fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 >> 172.21.15.3:0/4144289937 conn(0x7fe190078a10 msgr2=0x7fe190100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 shutdown_connections 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:51.101+0000 7fe196a28700 1 -- 172.21.15.3:0/4144289937 wait complete. 2024-05-08T05:53:51.106 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:53:51.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.410 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi089 to 176.1M 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: Unable to set osd_memory_target on smithi089 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.411 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:51 smithi003 ceph-mon[30137]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi089 to 176.1M 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: Unable to set osd_memory_target on smithi089 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:51 smithi089 ceph-mon[34495]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi089 to 176.1M 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: Unable to set osd_memory_target on smithi089 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:51.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:51 smithi104 ceph-mon[34559]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:53:51.900 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-05-08T05:53:50.524016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188055,"num_objects":28,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":521,"num_read_kb":395,"num_write":335,"num_write_kb":3568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":751,"ondisk_log_size":751,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361296,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580528,"statfs":{"total":1151940427776,"available":1151570460672,"internally_reserved":0,"allocated":12394496,"data_stored":9817853,"data_compressed":601395,"data_compressed_allocated":815104,"data_compressed_original":4481024,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989333"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614721+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:53:46.614721+0000","last_peered":"2024-05-08T05:53:46.614721+0000","last_clean":"2024-05-08T05:53:46.614721+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:53:46.614721+0000","last_undegraded":"2024-05-08T05:53:46.614721+0000","last_fullsized":"2024-05-08T05:53:46.614721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614729+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:53:46.614729+0000","last_peered":"2024-05-08T05:53:46.614729+0000","last_clean":"2024-05-08T05:53:46.614729+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:53:46.614729+0000","last_undegraded":"2024-05-08T05:53:46.614729+0000","last_fullsized":"2024-05-08T05:53:46.614729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614488+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:53:46.614488+0000","last_peered":"2024-05-08T05:53:46.614488+0000","last_clean":"2024-05-08T05:53:46.614488+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:53:46.614488+0000","last_undegraded":"2024-05-08T05:53:46.614488+0000","last_fullsized":"2024-05-08T05:53:46.614488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614723+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:53:46.614723+0000","last_peered":"2024-05-08T05:53:46.614723+0000","last_clean":"2024-05-08T05:53:46.614723+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:53:46.614723+0000","last_undegraded":"2024-05-08T05:53:46.614723+0000","last_fullsized":"2024-05-08T05:53:46.614723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614919+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:53:46.614919+0000","last_peered":"2024-05-08T05:53:46.614919+0000","last_clean":"2024-05-08T05:53:46.614919+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:53:46.614919+0000","last_undegraded":"2024-05-08T05:53:46.614919+0000","last_fullsized":"2024-05-08T05:53:46.614919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615081+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:53:46.615081+0000","last_peered":"2024-05-08T05:53:46.615081+0000","last_clean":"2024-05-08T05:53:46.615081+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:53:46.615081+0000","last_undegraded":"2024-05-08T05:53:46.615081+0000","last_fullsized":"2024-05-08T05:53:46.615081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511387+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:53:46.511387+0000","last_peered":"2024-05-08T05:53:46.511387+0000","last_clean":"2024-05-08T05:53:46.511387+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:53:46.511387+0000","last_undegraded":"2024-05-08T05:53:46.511387+0000","last_fullsized":"2024-05-08T05:53:46.511387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614039+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:53:46.614039+0000","last_peered":"2024-05-08T05:53:46.614039+0000","last_clean":"2024-05-08T05:53:46.614039+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:53:46.614039+0000","last_undegraded":"2024-05-08T05:53:46.614039+0000","last_fullsized":"2024-05-08T05:53:46.614039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615286+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:53:46.615286+0000","last_peered":"2024-05-08T05:53:46.615286+0000","last_clean":"2024-05-08T05:53:46.615286+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:53:46.615286+0000","last_undegraded":"2024-05-08T05:53:46.615286+0000","last_fullsized":"2024-05-08T05:53:46.615286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615772+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:53:46.615772+0000","last_peered":"2024-05-08T05:53:46.615772+0000","last_clean":"2024-05-08T05:53:46.615772+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:53:46.615772+0000","last_undegraded":"2024-05-08T05:53:46.615772+0000","last_fullsized":"2024-05-08T05:53:46.615772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511639+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:53:46.511639+0000","last_peered":"2024-05-08T05:53:46.511639+0000","last_clean":"2024-05-08T05:53:46.511639+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:53:46.511639+0000","last_undegraded":"2024-05-08T05:53:46.511639+0000","last_fullsized":"2024-05-08T05:53:46.511639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614712+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:53:46.614712+0000","last_peered":"2024-05-08T05:53:46.614712+0000","last_clean":"2024-05-08T05:53:46.614712+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:53:46.614712+0000","last_undegraded":"2024-05-08T05:53:46.614712+0000","last_fullsized":"2024-05-08T05:53:46.614712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509926+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:53:46.509926+0000","last_peered":"2024-05-08T05:53:46.509926+0000","last_clean":"2024-05-08T05:53:46.509926+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:53:46.509926+0000","last_undegraded":"2024-05-08T05:53:46.509926+0000","last_fullsized":"2024-05-08T05:53:46.509926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614223+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:53:46.614223+0000","last_peered":"2024-05-08T05:53:46.614223+0000","last_clean":"2024-05-08T05:53:46.614223+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:53:46.614223+0000","last_undegraded":"2024-05-08T05:53:46.614223+0000","last_fullsized":"2024-05-08T05:53:46.614223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614004+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:53:46.614004+0000","last_peered":"2024-05-08T05:53:46.614004+0000","last_clean":"2024-05-08T05:53:46.614004+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:53:46.614004+0000","last_undegraded":"2024-05-08T05:53:46.614004+0000","last_fullsized":"2024-05-08T05:53:46.614004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613819+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:53:46.613819+0000","last_peered":"2024-05-08T05:53:46.613819+0000","last_clean":"2024-05-08T05:53:46.613819+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:53:46.613819+0000","last_undegraded":"2024-05-08T05:53:46.613819+0000","last_fullsized":"2024-05-08T05:53:46.613819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614378+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:53:46.614378+0000","last_peered":"2024-05-08T05:53:46.614378+0000","last_clean":"2024-05-08T05:53:46.614378+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:53:46.614378+0000","last_undegraded":"2024-05-08T05:53:46.614378+0000","last_fullsized":"2024-05-08T05:53:46.614378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617301+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:53:46.617301+0000","last_peered":"2024-05-08T05:53:46.617301+0000","last_clean":"2024-05-08T05:53:46.617301+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:53:46.617301+0000","last_undegraded":"2024-05-08T05:53:46.617301+0000","last_fullsized":"2024-05-08T05:53:46.617301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615183+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:53:46.615183+0000","last_peered":"2024-05-08T05:53:46.615183+0000","last_clean":"2024-05-08T05:53:46.615183+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:53:46.615183+0000","last_undegraded":"2024-05-08T05:53:46.615183+0000","last_fullsized":"2024-05-08T05:53:46.615183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614038+0000","last_change":"2024-05-08T05:51:35.736506+0000","last_active":"2024-05-08T05:53:46.614038+0000","last_peered":"2024-05-08T05:53:46.614038+0000","last_clean":"2024-05-08T05:53:46.614038+0000","last_became_active":"2024-05-08T05:51:35.735983+0000","last_became_peered":"2024-05-08T05:51:35.735983+0000","last_unstale":"2024-05-08T05:53:46.614038+0000","last_undegraded":"2024-05-08T05:53:46.614038+0000","last_fullsized":"2024-05-08T05:53:46.614038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:57.132249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617853+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:53:46.617853+0000","last_peered":"2024-05-08T05:53:46.617853+0000","last_clean":"2024-05-08T05:53:46.617853+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:53:46.617853+0000","last_undegraded":"2024-05-08T05:53:46.617853+0000","last_fullsized":"2024-05-08T05:53:46.617853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613902+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:53:46.613902+0000","last_peered":"2024-05-08T05:53:46.613902+0000","last_clean":"2024-05-08T05:53:46.613902+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:53:46.613902+0000","last_undegraded":"2024-05-08T05:53:46.613902+0000","last_fullsized":"2024-05-08T05:53:46.613902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613808+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:53:46.613808+0000","last_peered":"2024-05-08T05:53:46.613808+0000","last_clean":"2024-05-08T05:53:46.613808+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:53:46.613808+0000","last_undegraded":"2024-05-08T05:53:46.613808+0000","last_fullsized":"2024-05-08T05:53:46.613808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614162+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:53:46.614162+0000","last_peered":"2024-05-08T05:53:46.614162+0000","last_clean":"2024-05-08T05:53:46.614162+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:53:46.614162+0000","last_undegraded":"2024-05-08T05:53:46.614162+0000","last_fullsized":"2024-05-08T05:53:46.614162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615044+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:53:46.615044+0000","last_peered":"2024-05-08T05:53:46.615044+0000","last_clean":"2024-05-08T05:53:46.615044+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:53:46.615044+0000","last_undegraded":"2024-05-08T05:53:46.615044+0000","last_fullsized":"2024-05-08T05:53:46.615044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613891+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:53:46.613891+0000","last_peered":"2024-05-08T05:53:46.613891+0000","last_clean":"2024-05-08T05:53:46.613891+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:53:46.613891+0000","last_undegraded":"2024-05-08T05:53:46.613891+0000","last_fullsized":"2024-05-08T05:53:46.613891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614308+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:53:46.614308+0000","last_peered":"2024-05-08T05:53:46.614308+0000","last_clean":"2024-05-08T05:53:46.614308+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:53:46.614308+0000","last_undegraded":"2024-05-08T05:53:46.614308+0000","last_fullsized":"2024-05-08T05:53:46.614308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614346+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:53:46.614346+0000","last_peered":"2024-05-08T05:53:46.614346+0000","last_clean":"2024-05-08T05:53:46.614346+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:53:46.614346+0000","last_undegraded":"2024-05-08T05:53:46.614346+0000","last_fullsized":"2024-05-08T05:53:46.614346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614094+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:53:46.614094+0000","last_peered":"2024-05-08T05:53:46.614094+0000","last_clean":"2024-05-08T05:53:46.614094+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:53:46.614094+0000","last_undegraded":"2024-05-08T05:53:46.614094+0000","last_fullsized":"2024-05-08T05:53:46.614094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614430+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:53:46.614430+0000","last_peered":"2024-05-08T05:53:46.614430+0000","last_clean":"2024-05-08T05:53:46.614430+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:53:46.614430+0000","last_undegraded":"2024-05-08T05:53:46.614430+0000","last_fullsized":"2024-05-08T05:53:46.614430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513618+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:53:46.513618+0000","last_peered":"2024-05-08T05:53:46.513618+0000","last_clean":"2024-05-08T05:53:46.513618+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:53:46.513618+0000","last_undegraded":"2024-05-08T05:53:46.513618+0000","last_fullsized":"2024-05-08T05:53:46.513618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614897+0000","last_change":"2024-05-08T05:51:35.736946+0000","last_active":"2024-05-08T05:53:46.614897+0000","last_peered":"2024-05-08T05:53:46.614897+0000","last_clean":"2024-05-08T05:53:46.614897+0000","last_became_active":"2024-05-08T05:51:35.736199+0000","last_became_peered":"2024-05-08T05:51:35.736199+0000","last_unstale":"2024-05-08T05:53:46.614897+0000","last_undegraded":"2024-05-08T05:53:46.614897+0000","last_fullsized":"2024-05-08T05:53:46.614897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:47:09.503650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513623+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:53:46.513623+0000","last_peered":"2024-05-08T05:53:46.513623+0000","last_clean":"2024-05-08T05:53:46.513623+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:53:46.513623+0000","last_undegraded":"2024-05-08T05:53:46.513623+0000","last_fullsized":"2024-05-08T05:53:46.513623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614245+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:53:46.614245+0000","last_peered":"2024-05-08T05:53:46.614245+0000","last_clean":"2024-05-08T05:53:46.614245+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:53:46.614245+0000","last_undegraded":"2024-05-08T05:53:46.614245+0000","last_fullsized":"2024-05-08T05:53:46.614245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614461+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:53:46.614461+0000","last_peered":"2024-05-08T05:53:46.614461+0000","last_clean":"2024-05-08T05:53:46.614461+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:53:46.614461+0000","last_undegraded":"2024-05-08T05:53:46.614461+0000","last_fullsized":"2024-05-08T05:53:46.614461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614955+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:53:46.614955+0000","last_peered":"2024-05-08T05:53:46.614955+0000","last_clean":"2024-05-08T05:53:46.614955+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:53:46.614955+0000","last_undegraded":"2024-05-08T05:53:46.614955+0000","last_fullsized":"2024-05-08T05:53:46.614955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513689+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:53:46.513689+0000","last_peered":"2024-05-08T05:53:46.513689+0000","last_clean":"2024-05-08T05:53:46.513689+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:53:46.513689+0000","last_undegraded":"2024-05-08T05:53:46.513689+0000","last_fullsized":"2024-05-08T05:53:46.513689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614248+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:53:46.614248+0000","last_peered":"2024-05-08T05:53:46.614248+0000","last_clean":"2024-05-08T05:53:46.614248+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:53:46.614248+0000","last_undegraded":"2024-05-08T05:53:46.614248+0000","last_fullsized":"2024-05-08T05:53:46.614248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509660+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:53:46.509660+0000","last_peered":"2024-05-08T05:53:46.509660+0000","last_clean":"2024-05-08T05:53:46.509660+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:53:46.509660+0000","last_undegraded":"2024-05-08T05:53:46.509660+0000","last_fullsized":"2024-05-08T05:53:46.509660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614291+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:53:46.614291+0000","last_peered":"2024-05-08T05:53:46.614291+0000","last_clean":"2024-05-08T05:53:46.614291+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:53:46.614291+0000","last_undegraded":"2024-05-08T05:53:46.614291+0000","last_fullsized":"2024-05-08T05:53:46.614291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614723+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:53:46.614723+0000","last_peered":"2024-05-08T05:53:46.614723+0000","last_clean":"2024-05-08T05:53:46.614723+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:53:46.614723+0000","last_undegraded":"2024-05-08T05:53:46.614723+0000","last_fullsized":"2024-05-08T05:53:46.614723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513070+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:53:46.513070+0000","last_peered":"2024-05-08T05:53:46.513070+0000","last_clean":"2024-05-08T05:53:46.513070+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:53:46.513070+0000","last_undegraded":"2024-05-08T05:53:46.513070+0000","last_fullsized":"2024-05-08T05:53:46.513070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616751+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:53:46.616751+0000","last_peered":"2024-05-08T05:53:46.616751+0000","last_clean":"2024-05-08T05:53:46.616751+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:53:46.616751+0000","last_undegraded":"2024-05-08T05:53:46.616751+0000","last_fullsized":"2024-05-08T05:53:46.616751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614639+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:53:46.614639+0000","last_peered":"2024-05-08T05:53:46.614639+0000","last_clean":"2024-05-08T05:53:46.614639+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:53:46.614639+0000","last_undegraded":"2024-05-08T05:53:46.614639+0000","last_fullsized":"2024-05-08T05:53:46.614639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615870+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:53:46.615870+0000","last_peered":"2024-05-08T05:53:46.615870+0000","last_clean":"2024-05-08T05:53:46.615870+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:53:46.615870+0000","last_undegraded":"2024-05-08T05:53:46.615870+0000","last_fullsized":"2024-05-08T05:53:46.615870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615261+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:53:46.615261+0000","last_peered":"2024-05-08T05:53:46.615261+0000","last_clean":"2024-05-08T05:53:46.615261+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:53:46.615261+0000","last_undegraded":"2024-05-08T05:53:46.615261+0000","last_fullsized":"2024-05-08T05:53:46.615261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616532+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:53:46.616532+0000","last_peered":"2024-05-08T05:53:46.616532+0000","last_clean":"2024-05-08T05:53:46.616532+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:53:46.616532+0000","last_undegraded":"2024-05-08T05:53:46.616532+0000","last_fullsized":"2024-05-08T05:53:46.616532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511819+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:53:46.511819+0000","last_peered":"2024-05-08T05:53:46.511819+0000","last_clean":"2024-05-08T05:53:46.511819+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:53:46.511819+0000","last_undegraded":"2024-05-08T05:53:46.511819+0000","last_fullsized":"2024-05-08T05:53:46.511819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614831+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:53:46.614831+0000","last_peered":"2024-05-08T05:53:46.614831+0000","last_clean":"2024-05-08T05:53:46.614831+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:53:46.614831+0000","last_undegraded":"2024-05-08T05:53:46.614831+0000","last_fullsized":"2024-05-08T05:53:46.614831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614222+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:53:46.614222+0000","last_peered":"2024-05-08T05:53:46.614222+0000","last_clean":"2024-05-08T05:53:46.614222+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:53:46.614222+0000","last_undegraded":"2024-05-08T05:53:46.614222+0000","last_fullsized":"2024-05-08T05:53:46.614222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513588+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:53:46.513588+0000","last_peered":"2024-05-08T05:53:46.513588+0000","last_clean":"2024-05-08T05:53:46.513588+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:53:46.513588+0000","last_undegraded":"2024-05-08T05:53:46.513588+0000","last_fullsized":"2024-05-08T05:53:46.513588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614838+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:53:46.614838+0000","last_peered":"2024-05-08T05:53:46.614838+0000","last_clean":"2024-05-08T05:53:46.614838+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:53:46.614838+0000","last_undegraded":"2024-05-08T05:53:46.614838+0000","last_fullsized":"2024-05-08T05:53:46.614838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614704+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:53:46.614704+0000","last_peered":"2024-05-08T05:53:46.614704+0000","last_clean":"2024-05-08T05:53:46.614704+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:53:46.614704+0000","last_undegraded":"2024-05-08T05:53:46.614704+0000","last_fullsized":"2024-05-08T05:53:46.614704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":430,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.810400+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:53:46.810400+0000","last_peered":"2024-05-08T05:53:46.810400+0000","last_clean":"2024-05-08T05:53:46.810400+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:53:46.810400+0000","last_undegraded":"2024-05-08T05:53:46.810400+0000","last_fullsized":"2024-05-08T05:53:46.810400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71839,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":342,"num_read_kb":241,"num_write":41,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614614+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:53:46.614614+0000","last_peered":"2024-05-08T05:53:46.614614+0000","last_clean":"2024-05-08T05:53:46.614614+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:53:46.614614+0000","last_undegraded":"2024-05-08T05:53:46.614614+0000","last_fullsized":"2024-05-08T05:53:46.614614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513963+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:53:46.513963+0000","last_peered":"2024-05-08T05:53:46.513963+0000","last_clean":"2024-05-08T05:53:46.513963+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:53:46.513963+0000","last_undegraded":"2024-05-08T05:53:46.513963+0000","last_fullsized":"2024-05-08T05:53:46.513963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614240+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:53:46.614240+0000","last_peered":"2024-05-08T05:53:46.614240+0000","last_clean":"2024-05-08T05:53:46.614240+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:53:46.614240+0000","last_undegraded":"2024-05-08T05:53:46.614240+0000","last_fullsized":"2024-05-08T05:53:46.614240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509653+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:53:46.509653+0000","last_peered":"2024-05-08T05:53:46.509653+0000","last_clean":"2024-05-08T05:53:46.509653+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:53:46.509653+0000","last_undegraded":"2024-05-08T05:53:46.509653+0000","last_fullsized":"2024-05-08T05:53:46.509653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615055+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:53:46.615055+0000","last_peered":"2024-05-08T05:53:46.615055+0000","last_clean":"2024-05-08T05:53:46.615055+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:53:46.615055+0000","last_undegraded":"2024-05-08T05:53:46.615055+0000","last_fullsized":"2024-05-08T05:53:46.615055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513354+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:53:46.513354+0000","last_peered":"2024-05-08T05:53:46.513354+0000","last_clean":"2024-05-08T05:53:46.513354+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:53:46.513354+0000","last_undegraded":"2024-05-08T05:53:46.513354+0000","last_fullsized":"2024-05-08T05:53:46.513354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511440+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:53:46.511440+0000","last_peered":"2024-05-08T05:53:46.511440+0000","last_clean":"2024-05-08T05:53:46.511440+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:53:46.511440+0000","last_undegraded":"2024-05-08T05:53:46.511440+0000","last_fullsized":"2024-05-08T05:53:46.511440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617224+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:53:46.617224+0000","last_peered":"2024-05-08T05:53:46.617224+0000","last_clean":"2024-05-08T05:53:46.617224+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:53:46.617224+0000","last_undegraded":"2024-05-08T05:53:46.617224+0000","last_fullsized":"2024-05-08T05:53:46.617224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613969+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:53:46.613969+0000","last_peered":"2024-05-08T05:53:46.613969+0000","last_clean":"2024-05-08T05:53:46.613969+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:53:46.613969+0000","last_undegraded":"2024-05-08T05:53:46.613969+0000","last_fullsized":"2024-05-08T05:53:46.613969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511462+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:53:46.511462+0000","last_peered":"2024-05-08T05:53:46.511462+0000","last_clean":"2024-05-08T05:53:46.511462+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:53:46.511462+0000","last_undegraded":"2024-05-08T05:53:46.511462+0000","last_fullsized":"2024-05-08T05:53:46.511462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510363+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:53:46.510363+0000","last_peered":"2024-05-08T05:53:46.510363+0000","last_clean":"2024-05-08T05:53:46.510363+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:53:46.510363+0000","last_undegraded":"2024-05-08T05:53:46.510363+0000","last_fullsized":"2024-05-08T05:53:46.510363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614616+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:53:46.614616+0000","last_peered":"2024-05-08T05:53:46.614616+0000","last_clean":"2024-05-08T05:53:46.614616+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:53:46.614616+0000","last_undegraded":"2024-05-08T05:53:46.614616+0000","last_fullsized":"2024-05-08T05:53:46.614616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513265+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:53:46.513265+0000","last_peered":"2024-05-08T05:53:46.513265+0000","last_clean":"2024-05-08T05:53:46.513265+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:53:46.513265+0000","last_undegraded":"2024-05-08T05:53:46.513265+0000","last_fullsized":"2024-05-08T05:53:46.513265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513186+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:53:46.513186+0000","last_peered":"2024-05-08T05:53:46.513186+0000","last_clean":"2024-05-08T05:53:46.513186+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:53:46.513186+0000","last_undegraded":"2024-05-08T05:53:46.513186+0000","last_fullsized":"2024-05-08T05:53:46.513186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615070+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:53:46.615070+0000","last_peered":"2024-05-08T05:53:46.615070+0000","last_clean":"2024-05-08T05:53:46.615070+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:53:46.615070+0000","last_undegraded":"2024-05-08T05:53:46.615070+0000","last_fullsized":"2024-05-08T05:53:46.615070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614749+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:53:46.614749+0000","last_peered":"2024-05-08T05:53:46.614749+0000","last_clean":"2024-05-08T05:53:46.614749+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:53:46.614749+0000","last_undegraded":"2024-05-08T05:53:46.614749+0000","last_fullsized":"2024-05-08T05:53:46.614749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614135+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:53:46.614135+0000","last_peered":"2024-05-08T05:53:46.614135+0000","last_clean":"2024-05-08T05:53:46.614135+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:53:46.614135+0000","last_undegraded":"2024-05-08T05:53:46.614135+0000","last_fullsized":"2024-05-08T05:53:46.614135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617130+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:53:46.617130+0000","last_peered":"2024-05-08T05:53:46.617130+0000","last_clean":"2024-05-08T05:53:46.617130+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:53:46.617130+0000","last_undegraded":"2024-05-08T05:53:46.617130+0000","last_fullsized":"2024-05-08T05:53:46.617130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615364+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:53:46.615364+0000","last_peered":"2024-05-08T05:53:46.615364+0000","last_clean":"2024-05-08T05:53:46.615364+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:53:46.615364+0000","last_undegraded":"2024-05-08T05:53:46.615364+0000","last_fullsized":"2024-05-08T05:53:46.615364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510122+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:53:46.510122+0000","last_peered":"2024-05-08T05:53:46.510122+0000","last_clean":"2024-05-08T05:53:46.510122+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:53:46.510122+0000","last_undegraded":"2024-05-08T05:53:46.510122+0000","last_fullsized":"2024-05-08T05:53:46.510122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511385+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:53:46.511385+0000","last_peered":"2024-05-08T05:53:46.511385+0000","last_clean":"2024-05-08T05:53:46.511385+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:53:46.511385+0000","last_undegraded":"2024-05-08T05:53:46.511385+0000","last_fullsized":"2024-05-08T05:53:46.511385+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615576+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:53:46.615576+0000","last_peered":"2024-05-08T05:53:46.615576+0000","last_clean":"2024-05-08T05:53:46.615576+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:53:46.615576+0000","last_undegraded":"2024-05-08T05:53:46.615576+0000","last_fullsized":"2024-05-08T05:53:46.615576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510472+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:53:46.510472+0000","last_peered":"2024-05-08T05:53:46.510472+0000","last_clean":"2024-05-08T05:53:46.510472+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:53:46.510472+0000","last_undegraded":"2024-05-08T05:53:46.510472+0000","last_fullsized":"2024-05-08T05:53:46.510472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513347+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:53:46.513347+0000","last_peered":"2024-05-08T05:53:46.513347+0000","last_clean":"2024-05-08T05:53:46.513347+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:53:46.513347+0000","last_undegraded":"2024-05-08T05:53:46.513347+0000","last_fullsized":"2024-05-08T05:53:46.513347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509535+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:53:46.509535+0000","last_peered":"2024-05-08T05:53:46.509535+0000","last_clean":"2024-05-08T05:53:46.509535+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:53:46.509535+0000","last_undegraded":"2024-05-08T05:53:46.509535+0000","last_fullsized":"2024-05-08T05:53:46.509535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511414+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:53:46.511414+0000","last_peered":"2024-05-08T05:53:46.511414+0000","last_clean":"2024-05-08T05:53:46.511414+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:53:46.511414+0000","last_undegraded":"2024-05-08T05:53:46.511414+0000","last_fullsized":"2024-05-08T05:53:46.511414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615039+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:53:46.615039+0000","last_peered":"2024-05-08T05:53:46.615039+0000","last_clean":"2024-05-08T05:53:46.615039+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:53:46.615039+0000","last_undegraded":"2024-05-08T05:53:46.615039+0000","last_fullsized":"2024-05-08T05:53:46.615039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614261+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:53:46.614261+0000","last_peered":"2024-05-08T05:53:46.614261+0000","last_clean":"2024-05-08T05:53:46.614261+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:53:46.614261+0000","last_undegraded":"2024-05-08T05:53:46.614261+0000","last_fullsized":"2024-05-08T05:53:46.614261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511209+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:53:46.511209+0000","last_peered":"2024-05-08T05:53:46.511209+0000","last_clean":"2024-05-08T05:53:46.511209+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:53:46.511209+0000","last_undegraded":"2024-05-08T05:53:46.511209+0000","last_fullsized":"2024-05-08T05:53:46.511209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513641+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:53:46.513641+0000","last_peered":"2024-05-08T05:53:46.513641+0000","last_clean":"2024-05-08T05:53:46.513641+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:53:46.513641+0000","last_undegraded":"2024-05-08T05:53:46.513641+0000","last_fullsized":"2024-05-08T05:53:46.513641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509529+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:53:46.509529+0000","last_peered":"2024-05-08T05:53:46.509529+0000","last_clean":"2024-05-08T05:53:46.509529+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:53:46.509529+0000","last_undegraded":"2024-05-08T05:53:46.509529+0000","last_fullsized":"2024-05-08T05:53:46.509529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511399+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:53:46.511399+0000","last_peered":"2024-05-08T05:53:46.511399+0000","last_clean":"2024-05-08T05:53:46.511399+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:53:46.511399+0000","last_undegraded":"2024-05-08T05:53:46.511399+0000","last_fullsized":"2024-05-08T05:53:46.511399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614234+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:53:46.614234+0000","last_peered":"2024-05-08T05:53:46.614234+0000","last_clean":"2024-05-08T05:53:46.614234+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:53:46.614234+0000","last_undegraded":"2024-05-08T05:53:46.614234+0000","last_fullsized":"2024-05-08T05:53:46.614234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614508+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:53:46.614508+0000","last_peered":"2024-05-08T05:53:46.614508+0000","last_clean":"2024-05-08T05:53:46.614508+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:53:46.614508+0000","last_undegraded":"2024-05-08T05:53:46.614508+0000","last_fullsized":"2024-05-08T05:53:46.614508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510235+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:53:46.510235+0000","last_peered":"2024-05-08T05:53:46.510235+0000","last_clean":"2024-05-08T05:53:46.510235+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:53:46.510235+0000","last_undegraded":"2024-05-08T05:53:46.510235+0000","last_fullsized":"2024-05-08T05:53:46.510235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614195+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:53:46.614195+0000","last_peered":"2024-05-08T05:53:46.614195+0000","last_clean":"2024-05-08T05:53:46.614195+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:53:46.614195+0000","last_undegraded":"2024-05-08T05:53:46.614195+0000","last_fullsized":"2024-05-08T05:53:46.614195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'46","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.725739+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:53:46.725739+0000","last_peered":"2024-05-08T05:53:46.725739+0000","last_clean":"2024-05-08T05:53:46.725739+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:53:46.725739+0000","last_undegraded":"2024-05-08T05:53:46.725739+0000","last_fullsized":"2024-05-08T05:53:46.725739+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":83,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615918+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:53:46.615918+0000","last_peered":"2024-05-08T05:53:46.615918+0000","last_clean":"2024-05-08T05:53:46.615918+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:53:46.615918+0000","last_undegraded":"2024-05-08T05:53:46.615918+0000","last_fullsized":"2024-05-08T05:53:46.615918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613810+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:53:46.613810+0000","last_peered":"2024-05-08T05:53:46.613810+0000","last_clean":"2024-05-08T05:53:46.613810+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:53:46.613810+0000","last_undegraded":"2024-05-08T05:53:46.613810+0000","last_fullsized":"2024-05-08T05:53:46.613810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614825+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:53:46.614825+0000","last_peered":"2024-05-08T05:53:46.614825+0000","last_clean":"2024-05-08T05:53:46.614825+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:53:46.614825+0000","last_undegraded":"2024-05-08T05:53:46.614825+0000","last_fullsized":"2024-05-08T05:53:46.614825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614833+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:53:46.614833+0000","last_peered":"2024-05-08T05:53:46.614833+0000","last_clean":"2024-05-08T05:53:46.614833+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:53:46.614833+0000","last_undegraded":"2024-05-08T05:53:46.614833+0000","last_fullsized":"2024-05-08T05:53:46.614833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614454+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:53:46.614454+0000","last_peered":"2024-05-08T05:53:46.614454+0000","last_clean":"2024-05-08T05:53:46.614454+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:53:46.614454+0000","last_undegraded":"2024-05-08T05:53:46.614454+0000","last_fullsized":"2024-05-08T05:53:46.614454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613822+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:53:46.613822+0000","last_peered":"2024-05-08T05:53:46.613822+0000","last_clean":"2024-05-08T05:53:46.613822+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:53:46.613822+0000","last_undegraded":"2024-05-08T05:53:46.613822+0000","last_fullsized":"2024-05-08T05:53:46.613822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613910+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:53:46.613910+0000","last_peered":"2024-05-08T05:53:46.613910+0000","last_clean":"2024-05-08T05:53:46.613910+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:53:46.613910+0000","last_undegraded":"2024-05-08T05:53:46.613910+0000","last_fullsized":"2024-05-08T05:53:46.613910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617785+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:53:46.617785+0000","last_peered":"2024-05-08T05:53:46.617785+0000","last_clean":"2024-05-08T05:53:46.617785+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:53:46.617785+0000","last_undegraded":"2024-05-08T05:53:46.617785+0000","last_fullsized":"2024-05-08T05:53:46.617785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614741+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:53:46.614741+0000","last_peered":"2024-05-08T05:53:46.614741+0000","last_clean":"2024-05-08T05:53:46.614741+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:53:46.614741+0000","last_undegraded":"2024-05-08T05:53:46.614741+0000","last_fullsized":"2024-05-08T05:53:46.614741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615035+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:53:46.615035+0000","last_peered":"2024-05-08T05:53:46.615035+0000","last_clean":"2024-05-08T05:53:46.615035+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:53:46.615035+0000","last_undegraded":"2024-05-08T05:53:46.615035+0000","last_fullsized":"2024-05-08T05:53:46.615035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614455+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:53:46.614455+0000","last_peered":"2024-05-08T05:53:46.614455+0000","last_clean":"2024-05-08T05:53:46.614455+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:53:46.614455+0000","last_undegraded":"2024-05-08T05:53:46.614455+0000","last_fullsized":"2024-05-08T05:53:46.614455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614015+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:53:46.614015+0000","last_peered":"2024-05-08T05:53:46.614015+0000","last_clean":"2024-05-08T05:53:46.614015+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:53:46.614015+0000","last_undegraded":"2024-05-08T05:53:46.614015+0000","last_fullsized":"2024-05-08T05:53:46.614015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615562+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:53:46.615562+0000","last_peered":"2024-05-08T05:53:46.615562+0000","last_clean":"2024-05-08T05:53:46.615562+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:53:46.615562+0000","last_undegraded":"2024-05-08T05:53:46.615562+0000","last_fullsized":"2024-05-08T05:53:46.615562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614622+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:53:46.614622+0000","last_peered":"2024-05-08T05:53:46.614622+0000","last_clean":"2024-05-08T05:53:46.614622+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:53:46.614622+0000","last_undegraded":"2024-05-08T05:53:46.614622+0000","last_fullsized":"2024-05-08T05:53:46.614622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613925+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:53:46.613925+0000","last_peered":"2024-05-08T05:53:46.613925+0000","last_clean":"2024-05-08T05:53:46.613925+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:53:46.613925+0000","last_undegraded":"2024-05-08T05:53:46.613925+0000","last_fullsized":"2024-05-08T05:53:46.613925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511863+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:53:46.511863+0000","last_peered":"2024-05-08T05:53:46.511863+0000","last_clean":"2024-05-08T05:53:46.511863+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:53:46.511863+0000","last_undegraded":"2024-05-08T05:53:46.511863+0000","last_fullsized":"2024-05-08T05:53:46.511863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614904+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:53:46.614904+0000","last_peered":"2024-05-08T05:53:46.614904+0000","last_clean":"2024-05-08T05:53:46.614904+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:53:46.614904+0000","last_undegraded":"2024-05-08T05:53:46.614904+0000","last_fullsized":"2024-05-08T05:53:46.614904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614702+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:53:46.614702+0000","last_peered":"2024-05-08T05:53:46.614702+0000","last_clean":"2024-05-08T05:53:46.614702+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:53:46.614702+0000","last_undegraded":"2024-05-08T05:53:46.614702+0000","last_fullsized":"2024-05-08T05:53:46.614702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511809+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:53:46.511809+0000","last_peered":"2024-05-08T05:53:46.511809+0000","last_clean":"2024-05-08T05:53:46.511809+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:53:46.511809+0000","last_undegraded":"2024-05-08T05:53:46.511809+0000","last_fullsized":"2024-05-08T05:53:46.511809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614223+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:53:46.614223+0000","last_peered":"2024-05-08T05:53:46.614223+0000","last_clean":"2024-05-08T05:53:46.614223+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:53:46.614223+0000","last_undegraded":"2024-05-08T05:53:46.614223+0000","last_fullsized":"2024-05-08T05:53:46.614223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.723705+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:53:46.723705+0000","last_peered":"2024-05-08T05:53:46.723705+0000","last_clean":"2024-05-08T05:53:46.723705+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:53:46.723705+0000","last_undegraded":"2024-05-08T05:53:46.723705+0000","last_fullsized":"2024-05-08T05:53:46.723705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511491+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:53:46.511491+0000","last_peered":"2024-05-08T05:53:46.511491+0000","last_clean":"2024-05-08T05:53:46.511491+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:53:46.511491+0000","last_undegraded":"2024-05-08T05:53:46.511491+0000","last_fullsized":"2024-05-08T05:53:46.511491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.512982+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:53:46.512982+0000","last_peered":"2024-05-08T05:53:46.512982+0000","last_clean":"2024-05-08T05:53:46.512982+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:53:46.512982+0000","last_undegraded":"2024-05-08T05:53:46.512982+0000","last_fullsized":"2024-05-08T05:53:46.512982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510399+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:53:46.510399+0000","last_peered":"2024-05-08T05:53:46.510399+0000","last_clean":"2024-05-08T05:53:46.510399+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:53:46.510399+0000","last_undegraded":"2024-05-08T05:53:46.510399+0000","last_fullsized":"2024-05-08T05:53:46.510399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.512099+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:53:46.512099+0000","last_peered":"2024-05-08T05:53:46.512099+0000","last_clean":"2024-05-08T05:53:46.512099+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:53:46.512099+0000","last_undegraded":"2024-05-08T05:53:46.512099+0000","last_fullsized":"2024-05-08T05:53:46.512099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615200+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:53:46.615200+0000","last_peered":"2024-05-08T05:53:46.615200+0000","last_clean":"2024-05-08T05:53:46.615200+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:53:46.615200+0000","last_undegraded":"2024-05-08T05:53:46.615200+0000","last_fullsized":"2024-05-08T05:53:46.615200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614303+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:53:46.614303+0000","last_peered":"2024-05-08T05:53:46.614303+0000","last_clean":"2024-05-08T05:53:46.614303+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:53:46.614303+0000","last_undegraded":"2024-05-08T05:53:46.614303+0000","last_fullsized":"2024-05-08T05:53:46.614303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613822+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:53:46.613822+0000","last_peered":"2024-05-08T05:53:46.613822+0000","last_clean":"2024-05-08T05:53:46.613822+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:53:46.613822+0000","last_undegraded":"2024-05-08T05:53:46.613822+0000","last_fullsized":"2024-05-08T05:53:46.613822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614418+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:53:46.614418+0000","last_peered":"2024-05-08T05:53:46.614418+0000","last_clean":"2024-05-08T05:53:46.614418+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:53:46.614418+0000","last_undegraded":"2024-05-08T05:53:46.614418+0000","last_fullsized":"2024-05-08T05:53:46.614418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510198+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:53:46.510198+0000","last_peered":"2024-05-08T05:53:46.510198+0000","last_clean":"2024-05-08T05:53:46.510198+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:53:46.510198+0000","last_undegraded":"2024-05-08T05:53:46.510198+0000","last_fullsized":"2024-05-08T05:53:46.510198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614774+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:53:46.614774+0000","last_peered":"2024-05-08T05:53:46.614774+0000","last_clean":"2024-05-08T05:53:46.614774+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:53:46.614774+0000","last_undegraded":"2024-05-08T05:53:46.614774+0000","last_fullsized":"2024-05-08T05:53:46.614774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615168+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:53:46.615168+0000","last_peered":"2024-05-08T05:53:46.615168+0000","last_clean":"2024-05-08T05:53:46.615168+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:53:46.615168+0000","last_undegraded":"2024-05-08T05:53:46.615168+0000","last_fullsized":"2024-05-08T05:53:46.615168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616073+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:53:46.616073+0000","last_peered":"2024-05-08T05:53:46.616073+0000","last_clean":"2024-05-08T05:53:46.616073+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:53:46.616073+0000","last_undegraded":"2024-05-08T05:53:46.616073+0000","last_fullsized":"2024-05-08T05:53:46.616073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513136+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:53:46.513136+0000","last_peered":"2024-05-08T05:53:46.513136+0000","last_clean":"2024-05-08T05:53:46.513136+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:53:46.513136+0000","last_undegraded":"2024-05-08T05:53:46.513136+0000","last_fullsized":"2024-05-08T05:53:46.513136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616577+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:53:46.616577+0000","last_peered":"2024-05-08T05:53:46.616577+0000","last_clean":"2024-05-08T05:53:46.616577+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:53:46.616577+0000","last_undegraded":"2024-05-08T05:53:46.616577+0000","last_fullsized":"2024-05-08T05:53:46.616577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615118+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:53:46.615118+0000","last_peered":"2024-05-08T05:53:46.615118+0000","last_clean":"2024-05-08T05:53:46.615118+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:53:46.615118+0000","last_undegraded":"2024-05-08T05:53:46.615118+0000","last_fullsized":"2024-05-08T05:53:46.615118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509824+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:53:46.509824+0000","last_peered":"2024-05-08T05:53:46.509824+0000","last_clean":"2024-05-08T05:53:46.509824+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:53:46.509824+0000","last_undegraded":"2024-05-08T05:53:46.509824+0000","last_fullsized":"2024-05-08T05:53:46.509824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615234+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:53:46.615234+0000","last_peered":"2024-05-08T05:53:46.615234+0000","last_clean":"2024-05-08T05:53:46.615234+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:53:46.615234+0000","last_undegraded":"2024-05-08T05:53:46.615234+0000","last_fullsized":"2024-05-08T05:53:46.615234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106929,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597453,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":440,"num_read_kb":326,"num_write":210,"num_write_kb":1416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":1792359,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710780,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":524274,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907010,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1314179,"data_compressed":77181,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070536,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":922168,"data_compressed":54528,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266765,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":918094,"data_compressed":54452,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462993,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":1119320,"data_compressed":71025,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626519,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":520178,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":592017,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":510402,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":581403,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1045098,"data_compressed":60684,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986274,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":524296,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822748,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1069056,"data_stored":1246424,"data_compressed":77181,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":203633,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:53:51.905 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:53:52.176 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:52.652 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.653 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:52.654 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:52 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:52.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:52.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:52 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:52.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:52 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:53.608 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi003 to 177.2M 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Unable to set osd_memory_target on smithi003 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.609 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:53.610 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi003 to 177.2M 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Unable to set osd_memory_target on smithi003 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:53.764 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi003 to 177.2M 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Unable to set osd_memory_target on smithi003 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi003:/etc/ceph/ceph.conf 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi089:/etc/ceph/ceph.conf 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi104:/etc/ceph/ceph.conf 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 90 KiB/s wr, 66 op/s 2024-05-08T05:53:53.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi003:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi089:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: Updating smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:53:53.772 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:53:53.773 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:53:53.773 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.167+0000 7f6e1b671700 1 -- 172.21.15.3:0/3683189234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14108850 msgr2=0x7f6e141103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 --2- 172.21.15.3:0/3683189234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14108850 0x7f6e141103d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00b0d0 tx=0x7f6e0c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 -- 172.21.15.3:0/3683189234 shutdown_connections 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 --2- 172.21.15.3:0/3683189234 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14108850 0x7f6e141103d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 --2- 172.21.15.3:0/3683189234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14103750 0x7f6e14103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 --2- 172.21.15.3:0/3683189234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e14104100 0x7f6e14108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.170 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 -- 172.21.15.3:0/3683189234 >> 172.21.15.3:0/3683189234 conn(0x7f6e1407a600 msgr2=0x7f6e1407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 -- 172.21.15.3:0/3683189234 shutdown_connections 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 -- 172.21.15.3:0/3683189234 wait complete. 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 Processor -- start 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.168+0000 7f6e1b671700 1 -- start start 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e14103750 0x7f6e1419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 0x7f6e1419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14108850 0x7f6e14199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6e14113640 con 0x7f6e14104100 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141134c0 con 0x7f6e14108850 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e141137c0 con 0x7f6e14103750 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 0x7f6e1419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 0x7f6e1419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44954/0 (socket says 172.21.15.3:44954) 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 -- 172.21.15.3:0/1277820771 learned_addr learned my addr 172.21.15.3:0/1277820771 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1940d700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e14103750 0x7f6e1419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:54.171 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e19c0e700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14108850 0x7f6e14199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 -- 172.21.15.3:0/1277820771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e14103750 msgr2=0x7f6e1419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e14103750 0x7f6e1419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 -- 172.21.15.3:0/1277820771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14108850 msgr2=0x7f6e14199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14108850 0x7f6e14199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 -- 172.21.15.3:0/1277820771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e14199fe0 con 0x7f6e14104100 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e19c0e700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14108850 0x7f6e14199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e18c0c700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 0x7f6e1419fc00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e9c0 tx=0x7f6e0400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0400cbd0 con 0x7f6e14104100 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1419a2d0 con 0x7f6e14104100 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.169+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e0400cd70 con 0x7f6e14104100 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.170+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e04005610 con 0x7f6e14104100 2024-05-08T05:53:54.172 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.170+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e141a6590 con 0x7f6e14104100 2024-05-08T05:53:54.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.170+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e1406a190 con 0x7f6e14104100 2024-05-08T05:53:54.173 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.172+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7f6e04004750 con 0x7f6e14104100 2024-05-08T05:53:54.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.173+0000 7f6e0a7fc700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6e000b2a80 0x7f6e000b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:54.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.173+0000 7f6e1940d700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6e000b2a80 0x7f6e000b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:54.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.173+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f6e040e3080 con 0x7f6e14104100 2024-05-08T05:53:54.174 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.173+0000 7f6e1940d700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6e000b2a80 0x7f6e000b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e1419a9c0 tx=0x7f6e10009260 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:54.176 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.176+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e04054df0 con 0x7f6e14104100 2024-05-08T05:53:54.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.387+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e140638e0 con 0x7f6e000b2a80 2024-05-08T05:53:54.397 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.397+0000 7f6e0a7fc700 1 -- 172.21.15.3:0/1277820771 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339590 (secure 0 0 0) 0x7f6e140638e0 con 0x7f6e000b2a80 2024-05-08T05:53:54.401 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6e000b2a80 msgr2=0x7f6e000b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6e000b2a80 0x7f6e000b4f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6e1419a9c0 tx=0x7f6e10009260 comp rx=0 tx=0).stop 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 msgr2=0x7f6e1419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 0x7f6e1419fc00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6e0400e9c0 tx=0x7f6e0400ecd0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 shutdown_connections 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e14108850 0x7f6e14199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.404 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6e000b2a80 0x7f6e000b4f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e14103750 0x7f6e1419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 --2- 172.21.15.3:0/1277820771 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6e14104100 0x7f6e1419fc00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 >> 172.21.15.3:0/1277820771 conn(0x7f6e1407a600 msgr2=0x7f6e14101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 shutdown_connections 2024-05-08T05:53:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:54.402+0000 7f6e1b671700 1 -- 172.21.15.3:0/1277820771 wait complete. 2024-05-08T05:53:54.405 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:53:55.239 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-08T05:53:52.524364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188055,"num_objects":28,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":751,"num_read_kb":602,"num_write":436,"num_write_kb":4017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":805,"ondisk_log_size":805,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361296,"kb_used_data":12104,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580528,"statfs":{"total":1151940427776,"available":1151570460672,"internally_reserved":0,"allocated":12394496,"data_stored":9817853,"data_compressed":601538,"data_compressed_allocated":815104,"data_compressed_original":4481024,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":230,"num_read_kb":207,"num_write":101,"num_write_kb":449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.989680"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614721+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:53:46.614721+0000","last_peered":"2024-05-08T05:53:46.614721+0000","last_clean":"2024-05-08T05:53:46.614721+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:53:46.614721+0000","last_undegraded":"2024-05-08T05:53:46.614721+0000","last_fullsized":"2024-05-08T05:53:46.614721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614729+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:53:46.614729+0000","last_peered":"2024-05-08T05:53:46.614729+0000","last_clean":"2024-05-08T05:53:46.614729+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:53:46.614729+0000","last_undegraded":"2024-05-08T05:53:46.614729+0000","last_fullsized":"2024-05-08T05:53:46.614729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614488+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:53:46.614488+0000","last_peered":"2024-05-08T05:53:46.614488+0000","last_clean":"2024-05-08T05:53:46.614488+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:53:46.614488+0000","last_undegraded":"2024-05-08T05:53:46.614488+0000","last_fullsized":"2024-05-08T05:53:46.614488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614723+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:53:46.614723+0000","last_peered":"2024-05-08T05:53:46.614723+0000","last_clean":"2024-05-08T05:53:46.614723+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:53:46.614723+0000","last_undegraded":"2024-05-08T05:53:46.614723+0000","last_fullsized":"2024-05-08T05:53:46.614723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614919+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:53:46.614919+0000","last_peered":"2024-05-08T05:53:46.614919+0000","last_clean":"2024-05-08T05:53:46.614919+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:53:46.614919+0000","last_undegraded":"2024-05-08T05:53:46.614919+0000","last_fullsized":"2024-05-08T05:53:46.614919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615081+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:53:46.615081+0000","last_peered":"2024-05-08T05:53:46.615081+0000","last_clean":"2024-05-08T05:53:46.615081+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:53:46.615081+0000","last_undegraded":"2024-05-08T05:53:46.615081+0000","last_fullsized":"2024-05-08T05:53:46.615081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511387+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:53:46.511387+0000","last_peered":"2024-05-08T05:53:46.511387+0000","last_clean":"2024-05-08T05:53:46.511387+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:53:46.511387+0000","last_undegraded":"2024-05-08T05:53:46.511387+0000","last_fullsized":"2024-05-08T05:53:46.511387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614039+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:53:46.614039+0000","last_peered":"2024-05-08T05:53:46.614039+0000","last_clean":"2024-05-08T05:53:46.614039+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:53:46.614039+0000","last_undegraded":"2024-05-08T05:53:46.614039+0000","last_fullsized":"2024-05-08T05:53:46.614039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615286+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:53:46.615286+0000","last_peered":"2024-05-08T05:53:46.615286+0000","last_clean":"2024-05-08T05:53:46.615286+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:53:46.615286+0000","last_undegraded":"2024-05-08T05:53:46.615286+0000","last_fullsized":"2024-05-08T05:53:46.615286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615772+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:53:46.615772+0000","last_peered":"2024-05-08T05:53:46.615772+0000","last_clean":"2024-05-08T05:53:46.615772+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:53:46.615772+0000","last_undegraded":"2024-05-08T05:53:46.615772+0000","last_fullsized":"2024-05-08T05:53:46.615772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511639+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:53:46.511639+0000","last_peered":"2024-05-08T05:53:46.511639+0000","last_clean":"2024-05-08T05:53:46.511639+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:53:46.511639+0000","last_undegraded":"2024-05-08T05:53:46.511639+0000","last_fullsized":"2024-05-08T05:53:46.511639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614712+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:53:46.614712+0000","last_peered":"2024-05-08T05:53:46.614712+0000","last_clean":"2024-05-08T05:53:46.614712+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:53:46.614712+0000","last_undegraded":"2024-05-08T05:53:46.614712+0000","last_fullsized":"2024-05-08T05:53:46.614712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509926+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:53:46.509926+0000","last_peered":"2024-05-08T05:53:46.509926+0000","last_clean":"2024-05-08T05:53:46.509926+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:53:46.509926+0000","last_undegraded":"2024-05-08T05:53:46.509926+0000","last_fullsized":"2024-05-08T05:53:46.509926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614223+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:53:46.614223+0000","last_peered":"2024-05-08T05:53:46.614223+0000","last_clean":"2024-05-08T05:53:46.614223+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:53:46.614223+0000","last_undegraded":"2024-05-08T05:53:46.614223+0000","last_fullsized":"2024-05-08T05:53:46.614223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614004+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:53:46.614004+0000","last_peered":"2024-05-08T05:53:46.614004+0000","last_clean":"2024-05-08T05:53:46.614004+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:53:46.614004+0000","last_undegraded":"2024-05-08T05:53:46.614004+0000","last_fullsized":"2024-05-08T05:53:46.614004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613819+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:53:46.613819+0000","last_peered":"2024-05-08T05:53:46.613819+0000","last_clean":"2024-05-08T05:53:46.613819+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:53:46.613819+0000","last_undegraded":"2024-05-08T05:53:46.613819+0000","last_fullsized":"2024-05-08T05:53:46.613819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614378+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:53:46.614378+0000","last_peered":"2024-05-08T05:53:46.614378+0000","last_clean":"2024-05-08T05:53:46.614378+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:53:46.614378+0000","last_undegraded":"2024-05-08T05:53:46.614378+0000","last_fullsized":"2024-05-08T05:53:46.614378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617301+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:53:46.617301+0000","last_peered":"2024-05-08T05:53:46.617301+0000","last_clean":"2024-05-08T05:53:46.617301+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:53:46.617301+0000","last_undegraded":"2024-05-08T05:53:46.617301+0000","last_fullsized":"2024-05-08T05:53:46.617301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615183+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:53:46.615183+0000","last_peered":"2024-05-08T05:53:46.615183+0000","last_clean":"2024-05-08T05:53:46.615183+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:53:46.615183+0000","last_undegraded":"2024-05-08T05:53:46.615183+0000","last_fullsized":"2024-05-08T05:53:46.615183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614038+0000","last_change":"2024-05-08T05:51:35.736506+0000","last_active":"2024-05-08T05:53:46.614038+0000","last_peered":"2024-05-08T05:53:46.614038+0000","last_clean":"2024-05-08T05:53:46.614038+0000","last_became_active":"2024-05-08T05:51:35.735983+0000","last_became_peered":"2024-05-08T05:51:35.735983+0000","last_unstale":"2024-05-08T05:53:46.614038+0000","last_undegraded":"2024-05-08T05:53:46.614038+0000","last_fullsized":"2024-05-08T05:53:46.614038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:57.132249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617853+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:53:46.617853+0000","last_peered":"2024-05-08T05:53:46.617853+0000","last_clean":"2024-05-08T05:53:46.617853+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:53:46.617853+0000","last_undegraded":"2024-05-08T05:53:46.617853+0000","last_fullsized":"2024-05-08T05:53:46.617853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613902+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:53:46.613902+0000","last_peered":"2024-05-08T05:53:46.613902+0000","last_clean":"2024-05-08T05:53:46.613902+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:53:46.613902+0000","last_undegraded":"2024-05-08T05:53:46.613902+0000","last_fullsized":"2024-05-08T05:53:46.613902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613808+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:53:46.613808+0000","last_peered":"2024-05-08T05:53:46.613808+0000","last_clean":"2024-05-08T05:53:46.613808+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:53:46.613808+0000","last_undegraded":"2024-05-08T05:53:46.613808+0000","last_fullsized":"2024-05-08T05:53:46.613808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614162+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:53:46.614162+0000","last_peered":"2024-05-08T05:53:46.614162+0000","last_clean":"2024-05-08T05:53:46.614162+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:53:46.614162+0000","last_undegraded":"2024-05-08T05:53:46.614162+0000","last_fullsized":"2024-05-08T05:53:46.614162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615044+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:53:46.615044+0000","last_peered":"2024-05-08T05:53:46.615044+0000","last_clean":"2024-05-08T05:53:46.615044+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:53:46.615044+0000","last_undegraded":"2024-05-08T05:53:46.615044+0000","last_fullsized":"2024-05-08T05:53:46.615044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613891+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:53:46.613891+0000","last_peered":"2024-05-08T05:53:46.613891+0000","last_clean":"2024-05-08T05:53:46.613891+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:53:46.613891+0000","last_undegraded":"2024-05-08T05:53:46.613891+0000","last_fullsized":"2024-05-08T05:53:46.613891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614308+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:53:46.614308+0000","last_peered":"2024-05-08T05:53:46.614308+0000","last_clean":"2024-05-08T05:53:46.614308+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:53:46.614308+0000","last_undegraded":"2024-05-08T05:53:46.614308+0000","last_fullsized":"2024-05-08T05:53:46.614308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614346+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:53:46.614346+0000","last_peered":"2024-05-08T05:53:46.614346+0000","last_clean":"2024-05-08T05:53:46.614346+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:53:46.614346+0000","last_undegraded":"2024-05-08T05:53:46.614346+0000","last_fullsized":"2024-05-08T05:53:46.614346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614094+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:53:46.614094+0000","last_peered":"2024-05-08T05:53:46.614094+0000","last_clean":"2024-05-08T05:53:46.614094+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:53:46.614094+0000","last_undegraded":"2024-05-08T05:53:46.614094+0000","last_fullsized":"2024-05-08T05:53:46.614094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614430+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:53:46.614430+0000","last_peered":"2024-05-08T05:53:46.614430+0000","last_clean":"2024-05-08T05:53:46.614430+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:53:46.614430+0000","last_undegraded":"2024-05-08T05:53:46.614430+0000","last_fullsized":"2024-05-08T05:53:46.614430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513618+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:53:46.513618+0000","last_peered":"2024-05-08T05:53:46.513618+0000","last_clean":"2024-05-08T05:53:46.513618+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:53:46.513618+0000","last_undegraded":"2024-05-08T05:53:46.513618+0000","last_fullsized":"2024-05-08T05:53:46.513618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614897+0000","last_change":"2024-05-08T05:51:35.736946+0000","last_active":"2024-05-08T05:53:46.614897+0000","last_peered":"2024-05-08T05:53:46.614897+0000","last_clean":"2024-05-08T05:53:46.614897+0000","last_became_active":"2024-05-08T05:51:35.736199+0000","last_became_peered":"2024-05-08T05:51:35.736199+0000","last_unstale":"2024-05-08T05:53:46.614897+0000","last_undegraded":"2024-05-08T05:53:46.614897+0000","last_fullsized":"2024-05-08T05:53:46.614897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:47:09.503650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513623+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:53:46.513623+0000","last_peered":"2024-05-08T05:53:46.513623+0000","last_clean":"2024-05-08T05:53:46.513623+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:53:46.513623+0000","last_undegraded":"2024-05-08T05:53:46.513623+0000","last_fullsized":"2024-05-08T05:53:46.513623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614245+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:53:46.614245+0000","last_peered":"2024-05-08T05:53:46.614245+0000","last_clean":"2024-05-08T05:53:46.614245+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:53:46.614245+0000","last_undegraded":"2024-05-08T05:53:46.614245+0000","last_fullsized":"2024-05-08T05:53:46.614245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614461+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:53:46.614461+0000","last_peered":"2024-05-08T05:53:46.614461+0000","last_clean":"2024-05-08T05:53:46.614461+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:53:46.614461+0000","last_undegraded":"2024-05-08T05:53:46.614461+0000","last_fullsized":"2024-05-08T05:53:46.614461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614955+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:53:46.614955+0000","last_peered":"2024-05-08T05:53:46.614955+0000","last_clean":"2024-05-08T05:53:46.614955+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:53:46.614955+0000","last_undegraded":"2024-05-08T05:53:46.614955+0000","last_fullsized":"2024-05-08T05:53:46.614955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513689+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:53:46.513689+0000","last_peered":"2024-05-08T05:53:46.513689+0000","last_clean":"2024-05-08T05:53:46.513689+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:53:46.513689+0000","last_undegraded":"2024-05-08T05:53:46.513689+0000","last_fullsized":"2024-05-08T05:53:46.513689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614248+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:53:46.614248+0000","last_peered":"2024-05-08T05:53:46.614248+0000","last_clean":"2024-05-08T05:53:46.614248+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:53:46.614248+0000","last_undegraded":"2024-05-08T05:53:46.614248+0000","last_fullsized":"2024-05-08T05:53:46.614248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509660+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:53:46.509660+0000","last_peered":"2024-05-08T05:53:46.509660+0000","last_clean":"2024-05-08T05:53:46.509660+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:53:46.509660+0000","last_undegraded":"2024-05-08T05:53:46.509660+0000","last_fullsized":"2024-05-08T05:53:46.509660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614291+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:53:46.614291+0000","last_peered":"2024-05-08T05:53:46.614291+0000","last_clean":"2024-05-08T05:53:46.614291+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:53:46.614291+0000","last_undegraded":"2024-05-08T05:53:46.614291+0000","last_fullsized":"2024-05-08T05:53:46.614291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614723+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:53:46.614723+0000","last_peered":"2024-05-08T05:53:46.614723+0000","last_clean":"2024-05-08T05:53:46.614723+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:53:46.614723+0000","last_undegraded":"2024-05-08T05:53:46.614723+0000","last_fullsized":"2024-05-08T05:53:46.614723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513070+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:53:46.513070+0000","last_peered":"2024-05-08T05:53:46.513070+0000","last_clean":"2024-05-08T05:53:46.513070+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:53:46.513070+0000","last_undegraded":"2024-05-08T05:53:46.513070+0000","last_fullsized":"2024-05-08T05:53:46.513070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616751+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:53:46.616751+0000","last_peered":"2024-05-08T05:53:46.616751+0000","last_clean":"2024-05-08T05:53:46.616751+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:53:46.616751+0000","last_undegraded":"2024-05-08T05:53:46.616751+0000","last_fullsized":"2024-05-08T05:53:46.616751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614639+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:53:46.614639+0000","last_peered":"2024-05-08T05:53:46.614639+0000","last_clean":"2024-05-08T05:53:46.614639+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:53:46.614639+0000","last_undegraded":"2024-05-08T05:53:46.614639+0000","last_fullsized":"2024-05-08T05:53:46.614639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615870+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:53:46.615870+0000","last_peered":"2024-05-08T05:53:46.615870+0000","last_clean":"2024-05-08T05:53:46.615870+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:53:46.615870+0000","last_undegraded":"2024-05-08T05:53:46.615870+0000","last_fullsized":"2024-05-08T05:53:46.615870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615261+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:53:46.615261+0000","last_peered":"2024-05-08T05:53:46.615261+0000","last_clean":"2024-05-08T05:53:46.615261+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:53:46.615261+0000","last_undegraded":"2024-05-08T05:53:46.615261+0000","last_fullsized":"2024-05-08T05:53:46.615261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616532+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:53:46.616532+0000","last_peered":"2024-05-08T05:53:46.616532+0000","last_clean":"2024-05-08T05:53:46.616532+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:53:46.616532+0000","last_undegraded":"2024-05-08T05:53:46.616532+0000","last_fullsized":"2024-05-08T05:53:46.616532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511819+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:53:46.511819+0000","last_peered":"2024-05-08T05:53:46.511819+0000","last_clean":"2024-05-08T05:53:46.511819+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:53:46.511819+0000","last_undegraded":"2024-05-08T05:53:46.511819+0000","last_fullsized":"2024-05-08T05:53:46.511819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614831+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:53:46.614831+0000","last_peered":"2024-05-08T05:53:46.614831+0000","last_clean":"2024-05-08T05:53:46.614831+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:53:46.614831+0000","last_undegraded":"2024-05-08T05:53:46.614831+0000","last_fullsized":"2024-05-08T05:53:46.614831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614222+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:53:46.614222+0000","last_peered":"2024-05-08T05:53:46.614222+0000","last_clean":"2024-05-08T05:53:46.614222+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:53:46.614222+0000","last_undegraded":"2024-05-08T05:53:46.614222+0000","last_fullsized":"2024-05-08T05:53:46.614222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513588+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:53:46.513588+0000","last_peered":"2024-05-08T05:53:46.513588+0000","last_clean":"2024-05-08T05:53:46.513588+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:53:46.513588+0000","last_undegraded":"2024-05-08T05:53:46.513588+0000","last_fullsized":"2024-05-08T05:53:46.513588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614838+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:53:46.614838+0000","last_peered":"2024-05-08T05:53:46.614838+0000","last_clean":"2024-05-08T05:53:46.614838+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:53:46.614838+0000","last_undegraded":"2024-05-08T05:53:46.614838+0000","last_fullsized":"2024-05-08T05:53:46.614838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614704+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:53:46.614704+0000","last_peered":"2024-05-08T05:53:46.614704+0000","last_clean":"2024-05-08T05:53:46.614704+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:53:46.614704+0000","last_undegraded":"2024-05-08T05:53:46.614704+0000","last_fullsized":"2024-05-08T05:53:46.614704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":430,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.810400+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:53:46.810400+0000","last_peered":"2024-05-08T05:53:46.810400+0000","last_clean":"2024-05-08T05:53:46.810400+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:53:46.810400+0000","last_undegraded":"2024-05-08T05:53:46.810400+0000","last_fullsized":"2024-05-08T05:53:46.810400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71839,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":342,"num_read_kb":241,"num_write":41,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614614+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:53:46.614614+0000","last_peered":"2024-05-08T05:53:46.614614+0000","last_clean":"2024-05-08T05:53:46.614614+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:53:46.614614+0000","last_undegraded":"2024-05-08T05:53:46.614614+0000","last_fullsized":"2024-05-08T05:53:46.614614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513963+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:53:46.513963+0000","last_peered":"2024-05-08T05:53:46.513963+0000","last_clean":"2024-05-08T05:53:46.513963+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:53:46.513963+0000","last_undegraded":"2024-05-08T05:53:46.513963+0000","last_fullsized":"2024-05-08T05:53:46.513963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614240+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:53:46.614240+0000","last_peered":"2024-05-08T05:53:46.614240+0000","last_clean":"2024-05-08T05:53:46.614240+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:53:46.614240+0000","last_undegraded":"2024-05-08T05:53:46.614240+0000","last_fullsized":"2024-05-08T05:53:46.614240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509653+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:53:46.509653+0000","last_peered":"2024-05-08T05:53:46.509653+0000","last_clean":"2024-05-08T05:53:46.509653+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:53:46.509653+0000","last_undegraded":"2024-05-08T05:53:46.509653+0000","last_fullsized":"2024-05-08T05:53:46.509653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615055+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:53:46.615055+0000","last_peered":"2024-05-08T05:53:46.615055+0000","last_clean":"2024-05-08T05:53:46.615055+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:53:46.615055+0000","last_undegraded":"2024-05-08T05:53:46.615055+0000","last_fullsized":"2024-05-08T05:53:46.615055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513354+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:53:46.513354+0000","last_peered":"2024-05-08T05:53:46.513354+0000","last_clean":"2024-05-08T05:53:46.513354+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:53:46.513354+0000","last_undegraded":"2024-05-08T05:53:46.513354+0000","last_fullsized":"2024-05-08T05:53:46.513354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511440+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:53:46.511440+0000","last_peered":"2024-05-08T05:53:46.511440+0000","last_clean":"2024-05-08T05:53:46.511440+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:53:46.511440+0000","last_undegraded":"2024-05-08T05:53:46.511440+0000","last_fullsized":"2024-05-08T05:53:46.511440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617224+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:53:46.617224+0000","last_peered":"2024-05-08T05:53:46.617224+0000","last_clean":"2024-05-08T05:53:46.617224+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:53:46.617224+0000","last_undegraded":"2024-05-08T05:53:46.617224+0000","last_fullsized":"2024-05-08T05:53:46.617224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613969+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:53:46.613969+0000","last_peered":"2024-05-08T05:53:46.613969+0000","last_clean":"2024-05-08T05:53:46.613969+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:53:46.613969+0000","last_undegraded":"2024-05-08T05:53:46.613969+0000","last_fullsized":"2024-05-08T05:53:46.613969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511462+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:53:46.511462+0000","last_peered":"2024-05-08T05:53:46.511462+0000","last_clean":"2024-05-08T05:53:46.511462+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:53:46.511462+0000","last_undegraded":"2024-05-08T05:53:46.511462+0000","last_fullsized":"2024-05-08T05:53:46.511462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510363+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:53:46.510363+0000","last_peered":"2024-05-08T05:53:46.510363+0000","last_clean":"2024-05-08T05:53:46.510363+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:53:46.510363+0000","last_undegraded":"2024-05-08T05:53:46.510363+0000","last_fullsized":"2024-05-08T05:53:46.510363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614616+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:53:46.614616+0000","last_peered":"2024-05-08T05:53:46.614616+0000","last_clean":"2024-05-08T05:53:46.614616+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:53:46.614616+0000","last_undegraded":"2024-05-08T05:53:46.614616+0000","last_fullsized":"2024-05-08T05:53:46.614616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513265+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:53:46.513265+0000","last_peered":"2024-05-08T05:53:46.513265+0000","last_clean":"2024-05-08T05:53:46.513265+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:53:46.513265+0000","last_undegraded":"2024-05-08T05:53:46.513265+0000","last_fullsized":"2024-05-08T05:53:46.513265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513186+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:53:46.513186+0000","last_peered":"2024-05-08T05:53:46.513186+0000","last_clean":"2024-05-08T05:53:46.513186+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:53:46.513186+0000","last_undegraded":"2024-05-08T05:53:46.513186+0000","last_fullsized":"2024-05-08T05:53:46.513186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615070+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:53:46.615070+0000","last_peered":"2024-05-08T05:53:46.615070+0000","last_clean":"2024-05-08T05:53:46.615070+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:53:46.615070+0000","last_undegraded":"2024-05-08T05:53:46.615070+0000","last_fullsized":"2024-05-08T05:53:46.615070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614749+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:53:46.614749+0000","last_peered":"2024-05-08T05:53:46.614749+0000","last_clean":"2024-05-08T05:53:46.614749+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:53:46.614749+0000","last_undegraded":"2024-05-08T05:53:46.614749+0000","last_fullsized":"2024-05-08T05:53:46.614749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614135+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:53:46.614135+0000","last_peered":"2024-05-08T05:53:46.614135+0000","last_clean":"2024-05-08T05:53:46.614135+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:53:46.614135+0000","last_undegraded":"2024-05-08T05:53:46.614135+0000","last_fullsized":"2024-05-08T05:53:46.614135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617130+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:53:46.617130+0000","last_peered":"2024-05-08T05:53:46.617130+0000","last_clean":"2024-05-08T05:53:46.617130+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:53:46.617130+0000","last_undegraded":"2024-05-08T05:53:46.617130+0000","last_fullsized":"2024-05-08T05:53:46.617130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615364+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:53:46.615364+0000","last_peered":"2024-05-08T05:53:46.615364+0000","last_clean":"2024-05-08T05:53:46.615364+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:53:46.615364+0000","last_undegraded":"2024-05-08T05:53:46.615364+0000","last_fullsized":"2024-05-08T05:53:46.615364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510122+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:53:46.510122+0000","last_peered":"2024-05-08T05:53:46.510122+0000","last_clean":"2024-05-08T05:53:46.510122+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:53:46.510122+0000","last_undegraded":"2024-05-08T05:53:46.510122+0000","last_fullsized":"2024-05-08T05:53:46.510122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511385+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:53:46.511385+0000","last_peered":"2024-05-08T05:53:46.511385+0000","last_clean":"2024-05-08T05:53:46.511385+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:53:46.511385+0000","last_undegraded":"2024-05-08T05:53:46.511385+0000","last_fullsized":"2024-05-08T05:53:46.511385+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615576+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:53:46.615576+0000","last_peered":"2024-05-08T05:53:46.615576+0000","last_clean":"2024-05-08T05:53:46.615576+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:53:46.615576+0000","last_undegraded":"2024-05-08T05:53:46.615576+0000","last_fullsized":"2024-05-08T05:53:46.615576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510472+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:53:46.510472+0000","last_peered":"2024-05-08T05:53:46.510472+0000","last_clean":"2024-05-08T05:53:46.510472+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:53:46.510472+0000","last_undegraded":"2024-05-08T05:53:46.510472+0000","last_fullsized":"2024-05-08T05:53:46.510472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513347+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:53:46.513347+0000","last_peered":"2024-05-08T05:53:46.513347+0000","last_clean":"2024-05-08T05:53:46.513347+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:53:46.513347+0000","last_undegraded":"2024-05-08T05:53:46.513347+0000","last_fullsized":"2024-05-08T05:53:46.513347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509535+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:53:46.509535+0000","last_peered":"2024-05-08T05:53:46.509535+0000","last_clean":"2024-05-08T05:53:46.509535+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:53:46.509535+0000","last_undegraded":"2024-05-08T05:53:46.509535+0000","last_fullsized":"2024-05-08T05:53:46.509535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511414+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:53:46.511414+0000","last_peered":"2024-05-08T05:53:46.511414+0000","last_clean":"2024-05-08T05:53:46.511414+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:53:46.511414+0000","last_undegraded":"2024-05-08T05:53:46.511414+0000","last_fullsized":"2024-05-08T05:53:46.511414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615039+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:53:46.615039+0000","last_peered":"2024-05-08T05:53:46.615039+0000","last_clean":"2024-05-08T05:53:46.615039+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:53:46.615039+0000","last_undegraded":"2024-05-08T05:53:46.615039+0000","last_fullsized":"2024-05-08T05:53:46.615039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614261+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:53:46.614261+0000","last_peered":"2024-05-08T05:53:46.614261+0000","last_clean":"2024-05-08T05:53:46.614261+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:53:46.614261+0000","last_undegraded":"2024-05-08T05:53:46.614261+0000","last_fullsized":"2024-05-08T05:53:46.614261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":122,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511209+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:53:46.511209+0000","last_peered":"2024-05-08T05:53:46.511209+0000","last_clean":"2024-05-08T05:53:46.511209+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:53:46.511209+0000","last_undegraded":"2024-05-08T05:53:46.511209+0000","last_fullsized":"2024-05-08T05:53:46.511209+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513641+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:53:46.513641+0000","last_peered":"2024-05-08T05:53:46.513641+0000","last_clean":"2024-05-08T05:53:46.513641+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:53:46.513641+0000","last_undegraded":"2024-05-08T05:53:46.513641+0000","last_fullsized":"2024-05-08T05:53:46.513641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509529+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:53:46.509529+0000","last_peered":"2024-05-08T05:53:46.509529+0000","last_clean":"2024-05-08T05:53:46.509529+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:53:46.509529+0000","last_undegraded":"2024-05-08T05:53:46.509529+0000","last_fullsized":"2024-05-08T05:53:46.509529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511399+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:53:46.511399+0000","last_peered":"2024-05-08T05:53:46.511399+0000","last_clean":"2024-05-08T05:53:46.511399+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:53:46.511399+0000","last_undegraded":"2024-05-08T05:53:46.511399+0000","last_fullsized":"2024-05-08T05:53:46.511399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614234+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:53:46.614234+0000","last_peered":"2024-05-08T05:53:46.614234+0000","last_clean":"2024-05-08T05:53:46.614234+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:53:46.614234+0000","last_undegraded":"2024-05-08T05:53:46.614234+0000","last_fullsized":"2024-05-08T05:53:46.614234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614508+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:53:46.614508+0000","last_peered":"2024-05-08T05:53:46.614508+0000","last_clean":"2024-05-08T05:53:46.614508+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:53:46.614508+0000","last_undegraded":"2024-05-08T05:53:46.614508+0000","last_fullsized":"2024-05-08T05:53:46.614508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510235+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:53:46.510235+0000","last_peered":"2024-05-08T05:53:46.510235+0000","last_clean":"2024-05-08T05:53:46.510235+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:53:46.510235+0000","last_undegraded":"2024-05-08T05:53:46.510235+0000","last_fullsized":"2024-05-08T05:53:46.510235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614195+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:53:46.614195+0000","last_peered":"2024-05-08T05:53:46.614195+0000","last_clean":"2024-05-08T05:53:46.614195+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:53:46.614195+0000","last_undegraded":"2024-05-08T05:53:46.614195+0000","last_fullsized":"2024-05-08T05:53:46.614195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'100","reported_seq":239,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:48.737785+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:53:48.737785+0000","last_peered":"2024-05-08T05:53:48.737785+0000","last_clean":"2024-05-08T05:53:48.737785+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:53:48.737785+0000","last_undegraded":"2024-05-08T05:53:48.737785+0000","last_fullsized":"2024-05-08T05:53:48.737785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":184,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615918+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:53:46.615918+0000","last_peered":"2024-05-08T05:53:46.615918+0000","last_clean":"2024-05-08T05:53:46.615918+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:53:46.615918+0000","last_undegraded":"2024-05-08T05:53:46.615918+0000","last_fullsized":"2024-05-08T05:53:46.615918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613810+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:53:46.613810+0000","last_peered":"2024-05-08T05:53:46.613810+0000","last_clean":"2024-05-08T05:53:46.613810+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:53:46.613810+0000","last_undegraded":"2024-05-08T05:53:46.613810+0000","last_fullsized":"2024-05-08T05:53:46.613810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614825+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:53:46.614825+0000","last_peered":"2024-05-08T05:53:46.614825+0000","last_clean":"2024-05-08T05:53:46.614825+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:53:46.614825+0000","last_undegraded":"2024-05-08T05:53:46.614825+0000","last_fullsized":"2024-05-08T05:53:46.614825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614833+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:53:46.614833+0000","last_peered":"2024-05-08T05:53:46.614833+0000","last_clean":"2024-05-08T05:53:46.614833+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:53:46.614833+0000","last_undegraded":"2024-05-08T05:53:46.614833+0000","last_fullsized":"2024-05-08T05:53:46.614833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614454+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:53:46.614454+0000","last_peered":"2024-05-08T05:53:46.614454+0000","last_clean":"2024-05-08T05:53:46.614454+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:53:46.614454+0000","last_undegraded":"2024-05-08T05:53:46.614454+0000","last_fullsized":"2024-05-08T05:53:46.614454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613822+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:53:46.613822+0000","last_peered":"2024-05-08T05:53:46.613822+0000","last_clean":"2024-05-08T05:53:46.613822+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:53:46.613822+0000","last_undegraded":"2024-05-08T05:53:46.613822+0000","last_fullsized":"2024-05-08T05:53:46.613822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613910+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:53:46.613910+0000","last_peered":"2024-05-08T05:53:46.613910+0000","last_clean":"2024-05-08T05:53:46.613910+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:53:46.613910+0000","last_undegraded":"2024-05-08T05:53:46.613910+0000","last_fullsized":"2024-05-08T05:53:46.613910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617785+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:53:46.617785+0000","last_peered":"2024-05-08T05:53:46.617785+0000","last_clean":"2024-05-08T05:53:46.617785+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:53:46.617785+0000","last_undegraded":"2024-05-08T05:53:46.617785+0000","last_fullsized":"2024-05-08T05:53:46.617785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614741+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:53:46.614741+0000","last_peered":"2024-05-08T05:53:46.614741+0000","last_clean":"2024-05-08T05:53:46.614741+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:53:46.614741+0000","last_undegraded":"2024-05-08T05:53:46.614741+0000","last_fullsized":"2024-05-08T05:53:46.614741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615035+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:53:46.615035+0000","last_peered":"2024-05-08T05:53:46.615035+0000","last_clean":"2024-05-08T05:53:46.615035+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:53:46.615035+0000","last_undegraded":"2024-05-08T05:53:46.615035+0000","last_fullsized":"2024-05-08T05:53:46.615035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614455+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:53:46.614455+0000","last_peered":"2024-05-08T05:53:46.614455+0000","last_clean":"2024-05-08T05:53:46.614455+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:53:46.614455+0000","last_undegraded":"2024-05-08T05:53:46.614455+0000","last_fullsized":"2024-05-08T05:53:46.614455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614015+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:53:46.614015+0000","last_peered":"2024-05-08T05:53:46.614015+0000","last_clean":"2024-05-08T05:53:46.614015+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:53:46.614015+0000","last_undegraded":"2024-05-08T05:53:46.614015+0000","last_fullsized":"2024-05-08T05:53:46.614015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615562+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:53:46.615562+0000","last_peered":"2024-05-08T05:53:46.615562+0000","last_clean":"2024-05-08T05:53:46.615562+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:53:46.615562+0000","last_undegraded":"2024-05-08T05:53:46.615562+0000","last_fullsized":"2024-05-08T05:53:46.615562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614622+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:53:46.614622+0000","last_peered":"2024-05-08T05:53:46.614622+0000","last_clean":"2024-05-08T05:53:46.614622+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:53:46.614622+0000","last_undegraded":"2024-05-08T05:53:46.614622+0000","last_fullsized":"2024-05-08T05:53:46.614622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613925+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:53:46.613925+0000","last_peered":"2024-05-08T05:53:46.613925+0000","last_clean":"2024-05-08T05:53:46.613925+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:53:46.613925+0000","last_undegraded":"2024-05-08T05:53:46.613925+0000","last_fullsized":"2024-05-08T05:53:46.613925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511863+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:53:46.511863+0000","last_peered":"2024-05-08T05:53:46.511863+0000","last_clean":"2024-05-08T05:53:46.511863+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:53:46.511863+0000","last_undegraded":"2024-05-08T05:53:46.511863+0000","last_fullsized":"2024-05-08T05:53:46.511863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614904+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:53:46.614904+0000","last_peered":"2024-05-08T05:53:46.614904+0000","last_clean":"2024-05-08T05:53:46.614904+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:53:46.614904+0000","last_undegraded":"2024-05-08T05:53:46.614904+0000","last_fullsized":"2024-05-08T05:53:46.614904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614702+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:53:46.614702+0000","last_peered":"2024-05-08T05:53:46.614702+0000","last_clean":"2024-05-08T05:53:46.614702+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:53:46.614702+0000","last_undegraded":"2024-05-08T05:53:46.614702+0000","last_fullsized":"2024-05-08T05:53:46.614702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511809+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:53:46.511809+0000","last_peered":"2024-05-08T05:53:46.511809+0000","last_clean":"2024-05-08T05:53:46.511809+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:53:46.511809+0000","last_undegraded":"2024-05-08T05:53:46.511809+0000","last_fullsized":"2024-05-08T05:53:46.511809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614223+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:53:46.614223+0000","last_peered":"2024-05-08T05:53:46.614223+0000","last_clean":"2024-05-08T05:53:46.614223+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:53:46.614223+0000","last_undegraded":"2024-05-08T05:53:46.614223+0000","last_fullsized":"2024-05-08T05:53:46.614223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'39","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.723705+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:53:46.723705+0000","last_peered":"2024-05-08T05:53:46.723705+0000","last_clean":"2024-05-08T05:53:46.723705+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:53:46.723705+0000","last_undegraded":"2024-05-08T05:53:46.723705+0000","last_fullsized":"2024-05-08T05:53:46.723705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511491+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:53:46.511491+0000","last_peered":"2024-05-08T05:53:46.511491+0000","last_clean":"2024-05-08T05:53:46.511491+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:53:46.511491+0000","last_undegraded":"2024-05-08T05:53:46.511491+0000","last_fullsized":"2024-05-08T05:53:46.511491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.512982+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:53:46.512982+0000","last_peered":"2024-05-08T05:53:46.512982+0000","last_clean":"2024-05-08T05:53:46.512982+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:53:46.512982+0000","last_undegraded":"2024-05-08T05:53:46.512982+0000","last_fullsized":"2024-05-08T05:53:46.512982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510399+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:53:46.510399+0000","last_peered":"2024-05-08T05:53:46.510399+0000","last_clean":"2024-05-08T05:53:46.510399+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:53:46.510399+0000","last_undegraded":"2024-05-08T05:53:46.510399+0000","last_fullsized":"2024-05-08T05:53:46.510399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.512099+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:53:46.512099+0000","last_peered":"2024-05-08T05:53:46.512099+0000","last_clean":"2024-05-08T05:53:46.512099+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:53:46.512099+0000","last_undegraded":"2024-05-08T05:53:46.512099+0000","last_fullsized":"2024-05-08T05:53:46.512099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615200+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:53:46.615200+0000","last_peered":"2024-05-08T05:53:46.615200+0000","last_clean":"2024-05-08T05:53:46.615200+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:53:46.615200+0000","last_undegraded":"2024-05-08T05:53:46.615200+0000","last_fullsized":"2024-05-08T05:53:46.615200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614303+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:53:46.614303+0000","last_peered":"2024-05-08T05:53:46.614303+0000","last_clean":"2024-05-08T05:53:46.614303+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:53:46.614303+0000","last_undegraded":"2024-05-08T05:53:46.614303+0000","last_fullsized":"2024-05-08T05:53:46.614303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613822+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:53:46.613822+0000","last_peered":"2024-05-08T05:53:46.613822+0000","last_clean":"2024-05-08T05:53:46.613822+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:53:46.613822+0000","last_undegraded":"2024-05-08T05:53:46.613822+0000","last_fullsized":"2024-05-08T05:53:46.613822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614418+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:53:46.614418+0000","last_peered":"2024-05-08T05:53:46.614418+0000","last_clean":"2024-05-08T05:53:46.614418+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:53:46.614418+0000","last_undegraded":"2024-05-08T05:53:46.614418+0000","last_fullsized":"2024-05-08T05:53:46.614418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510198+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:53:46.510198+0000","last_peered":"2024-05-08T05:53:46.510198+0000","last_clean":"2024-05-08T05:53:46.510198+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:53:46.510198+0000","last_undegraded":"2024-05-08T05:53:46.510198+0000","last_fullsized":"2024-05-08T05:53:46.510198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614774+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:53:46.614774+0000","last_peered":"2024-05-08T05:53:46.614774+0000","last_clean":"2024-05-08T05:53:46.614774+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:53:46.614774+0000","last_undegraded":"2024-05-08T05:53:46.614774+0000","last_fullsized":"2024-05-08T05:53:46.614774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615168+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:53:46.615168+0000","last_peered":"2024-05-08T05:53:46.615168+0000","last_clean":"2024-05-08T05:53:46.615168+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:53:46.615168+0000","last_undegraded":"2024-05-08T05:53:46.615168+0000","last_fullsized":"2024-05-08T05:53:46.615168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616073+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:53:46.616073+0000","last_peered":"2024-05-08T05:53:46.616073+0000","last_clean":"2024-05-08T05:53:46.616073+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:53:46.616073+0000","last_undegraded":"2024-05-08T05:53:46.616073+0000","last_fullsized":"2024-05-08T05:53:46.616073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513136+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:53:46.513136+0000","last_peered":"2024-05-08T05:53:46.513136+0000","last_clean":"2024-05-08T05:53:46.513136+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:53:46.513136+0000","last_undegraded":"2024-05-08T05:53:46.513136+0000","last_fullsized":"2024-05-08T05:53:46.513136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616577+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:53:46.616577+0000","last_peered":"2024-05-08T05:53:46.616577+0000","last_clean":"2024-05-08T05:53:46.616577+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:53:46.616577+0000","last_undegraded":"2024-05-08T05:53:46.616577+0000","last_fullsized":"2024-05-08T05:53:46.616577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615118+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:53:46.615118+0000","last_peered":"2024-05-08T05:53:46.615118+0000","last_clean":"2024-05-08T05:53:46.615118+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:53:46.615118+0000","last_undegraded":"2024-05-08T05:53:46.615118+0000","last_fullsized":"2024-05-08T05:53:46.615118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509824+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:53:46.509824+0000","last_peered":"2024-05-08T05:53:46.509824+0000","last_clean":"2024-05-08T05:53:46.509824+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:53:46.509824+0000","last_undegraded":"2024-05-08T05:53:46.509824+0000","last_fullsized":"2024-05-08T05:53:46.509824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615234+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:53:46.615234+0000","last_peered":"2024-05-08T05:53:46.615234+0000","last_clean":"2024-05-08T05:53:46.615234+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:53:46.615234+0000","last_undegraded":"2024-05-08T05:53:46.615234+0000","last_fullsized":"2024-05-08T05:53:46.615234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106929,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597453,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":670,"num_read_kb":533,"num_write":311,"num_write_kb":1865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":1792359,"data_compressed":76049,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710780,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":524274,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907010,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1146880,"data_stored":1314179,"data_compressed":77181,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":922168,"data_compressed":54671,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266765,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":918094,"data_compressed":54452,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462993,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":1119320,"data_compressed":71025,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626519,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":520178,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":592017,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":974848,"data_stored":510402,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":581403,"data_compressed":32408,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1036288,"data_stored":1045098,"data_compressed":60684,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986274,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":524296,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822748,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1069056,"data_stored":1246424,"data_compressed":77181,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":524920,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":131782,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":203633,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:53:55.244 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:53:55.244 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:53:55.504 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:55.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:55 smithi003 ceph-mon[30137]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:55.879 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:55 smithi003 ceph-mon[30137]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 131 KiB/s wr, 66 op/s 2024-05-08T05:53:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:55 smithi089 ceph-mon[34495]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:55 smithi089 ceph-mon[34495]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 131 KiB/s wr, 66 op/s 2024-05-08T05:53:56.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:55 smithi104 ceph-mon[34559]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:56.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:55 smithi104 ceph-mon[34559]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 131 KiB/s wr, 66 op/s 2024-05-08T05:53:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 -- 172.21.15.3:0/2888042296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a501023d0 msgr2=0x7f4a50102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 --2- 172.21.15.3:0/2888042296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a501023d0 0x7f4a50102850 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009990 tx=0x7f4a44009ca0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 -- 172.21.15.3:0/2888042296 shutdown_connections 2024-05-08T05:53:57.345 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 --2- 172.21.15.3:0/2888042296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a50102d90 0x7f4a5010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 --2- 172.21.15.3:0/2888042296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a50108450 0x7f4a50108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 --2- 172.21.15.3:0/2888042296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a501023d0 0x7f4a50102850 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 -- 172.21.15.3:0/2888042296 >> 172.21.15.3:0/2888042296 conn(0x7f4a50078410 msgr2=0x7f4a50078820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 -- 172.21.15.3:0/2888042296 shutdown_connections 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.344+0000 7f4a556cc700 1 -- 172.21.15.3:0/2888042296 wait complete. 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.345+0000 7f4a556cc700 1 Processor -- start 2024-05-08T05:53:57.346 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.345+0000 7f4a556cc700 1 -- start start 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a556cc700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a501023d0 0x7f4a50104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a556cc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 0x7f4a501045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a556cc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a50108450 0x7f4a50104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a556cc700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a50112540 con 0x7f4a50102d90 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a556cc700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4a501123c0 con 0x7f4a501023d0 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a556cc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4a501126c0 con 0x7f4a50108450 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 0x7f4a501045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4effd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a501023d0 0x7f4a50104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 0x7f4a501045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44968/0 (socket says 172.21.15.3:44968) 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 -- 172.21.15.3:0/676774226 learned_addr learned my addr 172.21.15.3:0/676774226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:53:57.348 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4f7fe700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a50108450 0x7f4a50104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 -- 172.21.15.3:0/676774226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a50108450 msgr2=0x7f4a50104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a50108450 0x7f4a50104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 -- 172.21.15.3:0/676774226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a501023d0 msgr2=0x7f4a50104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a501023d0 0x7f4a50104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 -- 172.21.15.3:0/676774226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a501a80f0 con 0x7f4a50102d90 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4effd700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a501023d0 0x7f4a50104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.346+0000 7f4a4e7fc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 0x7f4a501045c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a44003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.347+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44027070 con 0x7f4a50102d90 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.347+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a501a8380 con 0x7f4a50102d90 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.347+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a44003680 con 0x7f4a50102d90 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.347+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44022c70 con 0x7f4a50102d90 2024-05-08T05:53:57.349 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.347+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a501a87e0 con 0x7f4a50102d90 2024-05-08T05:53:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.348+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a50103ac0 con 0x7f4a50102d90 2024-05-08T05:53:57.350 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.350+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 24) ==== 149827+0+0 (secure 0 0 0) 0x7f4a4402a070 con 0x7f4a50102d90 2024-05-08T05:53:57.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.351+0000 7f4a37fff700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a380b2a80 0x7f4a380b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:53:57.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.351+0000 7f4a4effd700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a380b2a80 0x7f4a380b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:53:57.351 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.351+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f4a440e3bb0 con 0x7f4a50102d90 2024-05-08T05:53:57.352 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.352+0000 7f4a4effd700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a380b2a80 0x7f4a380b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005950 tx=0x7f4a4000b410 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:53:57.355 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.355+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a44013070 con 0x7f4a50102d90 2024-05-08T05:53:57.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.573+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4a501071d0 con 0x7f4a380b2a80 2024-05-08T05:53:57.583 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.583+0000 7f4a37fff700 1 -- 172.21.15.3:0/676774226 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339607 (secure 0 0 0) 0x7f4a501071d0 con 0x7f4a380b2a80 2024-05-08T05:53:57.588 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:53:57.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.588+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a380b2a80 msgr2=0x7f4a380b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:57.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.588+0000 7f4a556cc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a380b2a80 0x7f4a380b4f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005950 tx=0x7f4a4000b410 comp rx=0 tx=0).stop 2024-05-08T05:53:57.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.588+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 msgr2=0x7f4a501045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:53:57.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.588+0000 7f4a556cc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 0x7f4a501045c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a44003a90 comp rx=0 tx=0).stop 2024-05-08T05:53:57.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 shutdown_connections 2024-05-08T05:53:57.590 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a380b2a80 0x7f4a380b4f40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a50108450 0x7f4a50104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a501023d0 0x7f4a50104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 --2- 172.21.15.3:0/676774226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a50102d90 0x7f4a501045c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 >> 172.21.15.3:0/676774226 conn(0x7f4a50078410 msgr2=0x7f4a5006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 shutdown_connections 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:53:57.589+0000 7f4a556cc700 1 -- 172.21.15.3:0/676774226 wait complete. 2024-05-08T05:53:57.591 INFO:teuthology.orchestra.run.smithi003.stderr:dumped all 2024-05-08T05:53:57.600 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:57 smithi003 ceph-mon[30137]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 KiB/s wr, 51 op/s 2024-05-08T05:53:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:57 smithi089 ceph-mon[34495]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 KiB/s wr, 51 op/s 2024-05-08T05:53:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:57 smithi104 ceph-mon[34559]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 102 KiB/s wr, 51 op/s 2024-05-08T05:53:58.370 INFO:teuthology.orchestra.run.smithi003.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-05-08T05:53:56.525451+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1384687,"num_objects":28,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":847,"num_read_kb":809,"num_write":475,"num_write_kb":4483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":838,"ondisk_log_size":838,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361408,"kb_used_data":12200,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580416,"statfs":{"total":1151940427776,"available":1151570345984,"internally_reserved":0,"allocated":12492800,"data_stored":10428263,"data_compressed":634986,"data_compressed_allocated":860160,"data_compressed_original":5050368,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":326,"num_read_kb":414,"num_write":140,"num_write_kb":915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.990768"},"pg_stats":[{"pgid":"3.3f","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614721+0000","last_change":"2024-05-08T05:49:59.639205+0000","last_active":"2024-05-08T05:53:46.614721+0000","last_peered":"2024-05-08T05:53:46.614721+0000","last_clean":"2024-05-08T05:53:46.614721+0000","last_became_active":"2024-05-08T05:49:59.638507+0000","last_became_peered":"2024-05-08T05:49:59.638507+0000","last_unstale":"2024-05-08T05:53:46.614721+0000","last_undegraded":"2024-05-08T05:53:46.614721+0000","last_fullsized":"2024-05-08T05:53:46.614721+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:58.130136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614729+0000","last_change":"2024-05-08T05:49:59.639029+0000","last_active":"2024-05-08T05:53:46.614729+0000","last_peered":"2024-05-08T05:53:46.614729+0000","last_clean":"2024-05-08T05:53:46.614729+0000","last_became_active":"2024-05-08T05:49:59.638707+0000","last_became_peered":"2024-05-08T05:49:59.638707+0000","last_unstale":"2024-05-08T05:53:46.614729+0000","last_undegraded":"2024-05-08T05:53:46.614729+0000","last_fullsized":"2024-05-08T05:53:46.614729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:56.417274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614488+0000","last_change":"2024-05-08T05:49:59.638781+0000","last_active":"2024-05-08T05:53:46.614488+0000","last_peered":"2024-05-08T05:53:46.614488+0000","last_clean":"2024-05-08T05:53:46.614488+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:53:46.614488+0000","last_undegraded":"2024-05-08T05:53:46.614488+0000","last_fullsized":"2024-05-08T05:53:46.614488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:44:10.757940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"89'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614723+0000","last_change":"2024-05-08T05:49:59.638313+0000","last_active":"2024-05-08T05:53:46.614723+0000","last_peered":"2024-05-08T05:53:46.614723+0000","last_clean":"2024-05-08T05:53:46.614723+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:53:46.614723+0000","last_undegraded":"2024-05-08T05:53:46.614723+0000","last_fullsized":"2024-05-08T05:53:46.614723+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:56:34.269246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614919+0000","last_change":"2024-05-08T05:49:59.649698+0000","last_active":"2024-05-08T05:53:46.614919+0000","last_peered":"2024-05-08T05:53:46.614919+0000","last_clean":"2024-05-08T05:53:46.614919+0000","last_became_active":"2024-05-08T05:49:59.649302+0000","last_became_peered":"2024-05-08T05:49:59.649302+0000","last_unstale":"2024-05-08T05:53:46.614919+0000","last_undegraded":"2024-05-08T05:53:46.614919+0000","last_fullsized":"2024-05-08T05:53:46.614919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:29.790157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615081+0000","last_change":"2024-05-08T05:49:59.644086+0000","last_active":"2024-05-08T05:53:46.615081+0000","last_peered":"2024-05-08T05:53:46.615081+0000","last_clean":"2024-05-08T05:53:46.615081+0000","last_became_active":"2024-05-08T05:49:59.643695+0000","last_became_peered":"2024-05-08T05:49:59.643695+0000","last_unstale":"2024-05-08T05:53:46.615081+0000","last_undegraded":"2024-05-08T05:53:46.615081+0000","last_fullsized":"2024-05-08T05:53:46.615081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:53.681235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511387+0000","last_change":"2024-05-08T05:49:59.643200+0000","last_active":"2024-05-08T05:53:46.511387+0000","last_peered":"2024-05-08T05:53:46.511387+0000","last_clean":"2024-05-08T05:53:46.511387+0000","last_became_active":"2024-05-08T05:49:59.642829+0000","last_became_peered":"2024-05-08T05:49:59.642829+0000","last_unstale":"2024-05-08T05:53:46.511387+0000","last_undegraded":"2024-05-08T05:53:46.511387+0000","last_fullsized":"2024-05-08T05:53:46.511387+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:33.164463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614039+0000","last_change":"2024-05-08T05:49:59.645353+0000","last_active":"2024-05-08T05:53:46.614039+0000","last_peered":"2024-05-08T05:53:46.614039+0000","last_clean":"2024-05-08T05:53:46.614039+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:53:46.614039+0000","last_undegraded":"2024-05-08T05:53:46.614039+0000","last_fullsized":"2024-05-08T05:53:46.614039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:34:35.396507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615286+0000","last_change":"2024-05-08T05:49:59.645986+0000","last_active":"2024-05-08T05:53:46.615286+0000","last_peered":"2024-05-08T05:53:46.615286+0000","last_clean":"2024-05-08T05:53:46.615286+0000","last_became_active":"2024-05-08T05:49:59.644699+0000","last_became_peered":"2024-05-08T05:49:59.644699+0000","last_unstale":"2024-05-08T05:53:46.615286+0000","last_undegraded":"2024-05-08T05:53:46.615286+0000","last_fullsized":"2024-05-08T05:53:46.615286+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:27.079317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"89'16","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615772+0000","last_change":"2024-05-08T05:49:59.646714+0000","last_active":"2024-05-08T05:53:46.615772+0000","last_peered":"2024-05-08T05:53:46.615772+0000","last_clean":"2024-05-08T05:53:46.615772+0000","last_became_active":"2024-05-08T05:49:59.641684+0000","last_became_peered":"2024-05-08T05:49:59.641684+0000","last_unstale":"2024-05-08T05:53:46.615772+0000","last_undegraded":"2024-05-08T05:53:46.615772+0000","last_fullsized":"2024-05-08T05:53:46.615772+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.107464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511639+0000","last_change":"2024-05-08T05:49:59.648669+0000","last_active":"2024-05-08T05:53:46.511639+0000","last_peered":"2024-05-08T05:53:46.511639+0000","last_clean":"2024-05-08T05:53:46.511639+0000","last_became_active":"2024-05-08T05:49:59.648239+0000","last_became_peered":"2024-05-08T05:49:59.648239+0000","last_unstale":"2024-05-08T05:53:46.511639+0000","last_undegraded":"2024-05-08T05:53:46.511639+0000","last_fullsized":"2024-05-08T05:53:46.511639+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:14:15.587323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"89'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614712+0000","last_change":"2024-05-08T05:49:59.649564+0000","last_active":"2024-05-08T05:53:46.614712+0000","last_peered":"2024-05-08T05:53:46.614712+0000","last_clean":"2024-05-08T05:53:46.614712+0000","last_became_active":"2024-05-08T05:49:59.648527+0000","last_became_peered":"2024-05-08T05:49:59.648527+0000","last_unstale":"2024-05-08T05:53:46.614712+0000","last_undegraded":"2024-05-08T05:53:46.614712+0000","last_fullsized":"2024-05-08T05:53:46.614712+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:42.246444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509926+0000","last_change":"2024-05-08T05:49:59.647647+0000","last_active":"2024-05-08T05:53:46.509926+0000","last_peered":"2024-05-08T05:53:46.509926+0000","last_clean":"2024-05-08T05:53:46.509926+0000","last_became_active":"2024-05-08T05:49:59.647314+0000","last_became_peered":"2024-05-08T05:49:59.647314+0000","last_unstale":"2024-05-08T05:53:46.509926+0000","last_undegraded":"2024-05-08T05:53:46.509926+0000","last_fullsized":"2024-05-08T05:53:46.509926+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:33:05.027820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614223+0000","last_change":"2024-05-08T05:49:59.638302+0000","last_active":"2024-05-08T05:53:46.614223+0000","last_peered":"2024-05-08T05:53:46.614223+0000","last_clean":"2024-05-08T05:53:46.614223+0000","last_became_active":"2024-05-08T05:49:59.637959+0000","last_became_peered":"2024-05-08T05:49:59.637959+0000","last_unstale":"2024-05-08T05:53:46.614223+0000","last_undegraded":"2024-05-08T05:53:46.614223+0000","last_fullsized":"2024-05-08T05:53:46.614223+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:09.058596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614004+0000","last_change":"2024-05-08T05:49:59.640723+0000","last_active":"2024-05-08T05:53:46.614004+0000","last_peered":"2024-05-08T05:53:46.614004+0000","last_clean":"2024-05-08T05:53:46.614004+0000","last_became_active":"2024-05-08T05:49:59.640343+0000","last_became_peered":"2024-05-08T05:49:59.640343+0000","last_unstale":"2024-05-08T05:53:46.614004+0000","last_undegraded":"2024-05-08T05:53:46.614004+0000","last_fullsized":"2024-05-08T05:53:46.614004+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:22.858964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613819+0000","last_change":"2024-05-08T05:49:59.644342+0000","last_active":"2024-05-08T05:53:46.613819+0000","last_peered":"2024-05-08T05:53:46.613819+0000","last_clean":"2024-05-08T05:53:46.613819+0000","last_became_active":"2024-05-08T05:49:59.643981+0000","last_became_peered":"2024-05-08T05:49:59.643981+0000","last_unstale":"2024-05-08T05:53:46.613819+0000","last_undegraded":"2024-05-08T05:53:46.613819+0000","last_fullsized":"2024-05-08T05:53:46.613819+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:33.609206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614378+0000","last_change":"2024-05-08T05:49:59.638936+0000","last_active":"2024-05-08T05:53:46.614378+0000","last_peered":"2024-05-08T05:53:46.614378+0000","last_clean":"2024-05-08T05:53:46.614378+0000","last_became_active":"2024-05-08T05:49:59.638008+0000","last_became_peered":"2024-05-08T05:49:59.638008+0000","last_unstale":"2024-05-08T05:53:46.614378+0000","last_undegraded":"2024-05-08T05:53:46.614378+0000","last_fullsized":"2024-05-08T05:53:46.614378+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:35.794259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"89'10","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617301+0000","last_change":"2024-05-08T05:49:59.639397+0000","last_active":"2024-05-08T05:53:46.617301+0000","last_peered":"2024-05-08T05:53:46.617301+0000","last_clean":"2024-05-08T05:53:46.617301+0000","last_became_active":"2024-05-08T05:49:59.639005+0000","last_became_peered":"2024-05-08T05:49:59.639005+0000","last_unstale":"2024-05-08T05:53:46.617301+0000","last_undegraded":"2024-05-08T05:53:46.617301+0000","last_fullsized":"2024-05-08T05:53:46.617301+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:43.598901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615183+0000","last_change":"2024-05-08T05:49:59.652063+0000","last_active":"2024-05-08T05:53:46.615183+0000","last_peered":"2024-05-08T05:53:46.615183+0000","last_clean":"2024-05-08T05:53:46.615183+0000","last_became_active":"2024-05-08T05:49:59.651721+0000","last_became_peered":"2024-05-08T05:49:59.651721+0000","last_unstale":"2024-05-08T05:53:46.615183+0000","last_undegraded":"2024-05-08T05:53:46.615183+0000","last_fullsized":"2024-05-08T05:53:46.615183+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:08:14.839722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614038+0000","last_change":"2024-05-08T05:51:35.736506+0000","last_active":"2024-05-08T05:53:46.614038+0000","last_peered":"2024-05-08T05:53:46.614038+0000","last_clean":"2024-05-08T05:53:46.614038+0000","last_became_active":"2024-05-08T05:51:35.735983+0000","last_became_peered":"2024-05-08T05:51:35.735983+0000","last_unstale":"2024-05-08T05:53:46.614038+0000","last_undegraded":"2024-05-08T05:53:46.614038+0000","last_fullsized":"2024-05-08T05:53:46.614038+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:53:57.132249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"78'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617853+0000","last_change":"2024-05-08T05:49:56.610797+0000","last_active":"2024-05-08T05:53:46.617853+0000","last_peered":"2024-05-08T05:53:46.617853+0000","last_clean":"2024-05-08T05:53:46.617853+0000","last_became_active":"2024-05-08T05:49:56.610351+0000","last_became_peered":"2024-05-08T05:49:56.610351+0000","last_unstale":"2024-05-08T05:53:46.617853+0000","last_undegraded":"2024-05-08T05:53:46.617853+0000","last_fullsized":"2024-05-08T05:53:46.617853+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:28.222650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613902+0000","last_change":"2024-05-08T05:49:59.637907+0000","last_active":"2024-05-08T05:53:46.613902+0000","last_peered":"2024-05-08T05:53:46.613902+0000","last_clean":"2024-05-08T05:53:46.613902+0000","last_became_active":"2024-05-08T05:49:59.637596+0000","last_became_peered":"2024-05-08T05:49:59.637596+0000","last_unstale":"2024-05-08T05:53:46.613902+0000","last_undegraded":"2024-05-08T05:53:46.613902+0000","last_fullsized":"2024-05-08T05:53:46.613902+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:25.406257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613808+0000","last_change":"2024-05-08T05:50:12.712494+0000","last_active":"2024-05-08T05:53:46.613808+0000","last_peered":"2024-05-08T05:53:46.613808+0000","last_clean":"2024-05-08T05:53:46.613808+0000","last_became_active":"2024-05-08T05:50:12.711961+0000","last_became_peered":"2024-05-08T05:50:12.711961+0000","last_unstale":"2024-05-08T05:53:46.613808+0000","last_undegraded":"2024-05-08T05:53:46.613808+0000","last_fullsized":"2024-05-08T05:53:46.613808+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:00:37.198724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614162+0000","last_change":"2024-05-08T05:49:56.626133+0000","last_active":"2024-05-08T05:53:46.614162+0000","last_peered":"2024-05-08T05:53:46.614162+0000","last_clean":"2024-05-08T05:53:46.614162+0000","last_became_active":"2024-05-08T05:49:56.625803+0000","last_became_peered":"2024-05-08T05:49:56.625803+0000","last_unstale":"2024-05-08T05:53:46.614162+0000","last_undegraded":"2024-05-08T05:53:46.614162+0000","last_fullsized":"2024-05-08T05:53:46.614162+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:37:51.235639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615044+0000","last_change":"2024-05-08T05:49:59.635255+0000","last_active":"2024-05-08T05:53:46.615044+0000","last_peered":"2024-05-08T05:53:46.615044+0000","last_clean":"2024-05-08T05:53:46.615044+0000","last_became_active":"2024-05-08T05:49:59.634927+0000","last_became_peered":"2024-05-08T05:49:59.634927+0000","last_unstale":"2024-05-08T05:53:46.615044+0000","last_undegraded":"2024-05-08T05:53:46.615044+0000","last_fullsized":"2024-05-08T05:53:46.615044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:36.500695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613891+0000","last_change":"2024-05-08T05:50:12.719820+0000","last_active":"2024-05-08T05:53:46.613891+0000","last_peered":"2024-05-08T05:53:46.613891+0000","last_clean":"2024-05-08T05:53:46.613891+0000","last_became_active":"2024-05-08T05:50:12.719329+0000","last_became_peered":"2024-05-08T05:50:12.719329+0000","last_unstale":"2024-05-08T05:53:46.613891+0000","last_undegraded":"2024-05-08T05:53:46.613891+0000","last_fullsized":"2024-05-08T05:53:46.613891+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:47.628481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"78'2","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614308+0000","last_change":"2024-05-08T05:49:56.612396+0000","last_active":"2024-05-08T05:53:46.614308+0000","last_peered":"2024-05-08T05:53:46.614308+0000","last_clean":"2024-05-08T05:53:46.614308+0000","last_became_active":"2024-05-08T05:49:56.611866+0000","last_became_peered":"2024-05-08T05:49:56.611866+0000","last_unstale":"2024-05-08T05:53:46.614308+0000","last_undegraded":"2024-05-08T05:53:46.614308+0000","last_fullsized":"2024-05-08T05:53:46.614308+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:51.460599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614346+0000","last_change":"2024-05-08T05:49:59.653006+0000","last_active":"2024-05-08T05:53:46.614346+0000","last_peered":"2024-05-08T05:53:46.614346+0000","last_clean":"2024-05-08T05:53:46.614346+0000","last_became_active":"2024-05-08T05:49:59.652679+0000","last_became_peered":"2024-05-08T05:49:59.652679+0000","last_unstale":"2024-05-08T05:53:46.614346+0000","last_undegraded":"2024-05-08T05:53:46.614346+0000","last_fullsized":"2024-05-08T05:53:46.614346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:43.108548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614094+0000","last_change":"2024-05-08T05:50:12.706099+0000","last_active":"2024-05-08T05:53:46.614094+0000","last_peered":"2024-05-08T05:53:46.614094+0000","last_clean":"2024-05-08T05:53:46.614094+0000","last_became_active":"2024-05-08T05:50:12.705666+0000","last_became_peered":"2024-05-08T05:50:12.705666+0000","last_unstale":"2024-05-08T05:53:46.614094+0000","last_undegraded":"2024-05-08T05:53:46.614094+0000","last_fullsized":"2024-05-08T05:53:46.614094+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:20.191035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"78'2","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614430+0000","last_change":"2024-05-08T05:50:34.313697+0000","last_active":"2024-05-08T05:53:46.614430+0000","last_peered":"2024-05-08T05:53:46.614430+0000","last_clean":"2024-05-08T05:53:46.614430+0000","last_became_active":"2024-05-08T05:50:34.310180+0000","last_became_peered":"2024-05-08T05:50:34.310180+0000","last_unstale":"2024-05-08T05:53:46.614430+0000","last_undegraded":"2024-05-08T05:53:46.614430+0000","last_fullsized":"2024-05-08T05:53:46.614430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:48.742432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"89'15","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513618+0000","last_change":"2024-05-08T05:50:34.383855+0000","last_active":"2024-05-08T05:53:46.513618+0000","last_peered":"2024-05-08T05:53:46.513618+0000","last_clean":"2024-05-08T05:53:46.513618+0000","last_became_active":"2024-05-08T05:50:34.383441+0000","last_became_peered":"2024-05-08T05:50:34.383441+0000","last_unstale":"2024-05-08T05:53:46.513618+0000","last_undegraded":"2024-05-08T05:53:46.513618+0000","last_fullsized":"2024-05-08T05:53:46.513618+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:45:31.246851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614897+0000","last_change":"2024-05-08T05:51:35.736946+0000","last_active":"2024-05-08T05:53:46.614897+0000","last_peered":"2024-05-08T05:53:46.614897+0000","last_clean":"2024-05-08T05:53:46.614897+0000","last_became_active":"2024-05-08T05:51:35.736199+0000","last_became_peered":"2024-05-08T05:51:35.736199+0000","last_unstale":"2024-05-08T05:53:46.614897+0000","last_undegraded":"2024-05-08T05:53:46.614897+0000","last_fullsized":"2024-05-08T05:53:46.614897+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:47:09.503650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513623+0000","last_change":"2024-05-08T05:49:56.614902+0000","last_active":"2024-05-08T05:53:46.513623+0000","last_peered":"2024-05-08T05:53:46.513623+0000","last_clean":"2024-05-08T05:53:46.513623+0000","last_became_active":"2024-05-08T05:49:56.614449+0000","last_became_peered":"2024-05-08T05:49:56.614449+0000","last_unstale":"2024-05-08T05:53:46.513623+0000","last_undegraded":"2024-05-08T05:53:46.513623+0000","last_fullsized":"2024-05-08T05:53:46.513623+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:01.114439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614245+0000","last_change":"2024-05-08T05:49:59.645119+0000","last_active":"2024-05-08T05:53:46.614245+0000","last_peered":"2024-05-08T05:53:46.614245+0000","last_clean":"2024-05-08T05:53:46.614245+0000","last_became_active":"2024-05-08T05:49:59.644717+0000","last_became_peered":"2024-05-08T05:49:59.644717+0000","last_unstale":"2024-05-08T05:53:46.614245+0000","last_undegraded":"2024-05-08T05:53:46.614245+0000","last_fullsized":"2024-05-08T05:53:46.614245+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:46:17.978231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614461+0000","last_change":"2024-05-08T05:50:12.712022+0000","last_active":"2024-05-08T05:53:46.614461+0000","last_peered":"2024-05-08T05:53:46.614461+0000","last_clean":"2024-05-08T05:53:46.614461+0000","last_became_active":"2024-05-08T05:50:12.711559+0000","last_became_peered":"2024-05-08T05:50:12.711559+0000","last_unstale":"2024-05-08T05:53:46.614461+0000","last_undegraded":"2024-05-08T05:53:46.614461+0000","last_fullsized":"2024-05-08T05:53:46.614461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:09:15.468769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"78'1","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614955+0000","last_change":"2024-05-08T05:49:56.610598+0000","last_active":"2024-05-08T05:53:46.614955+0000","last_peered":"2024-05-08T05:53:46.614955+0000","last_clean":"2024-05-08T05:53:46.614955+0000","last_became_active":"2024-05-08T05:49:56.610210+0000","last_became_peered":"2024-05-08T05:49:56.610210+0000","last_unstale":"2024-05-08T05:53:46.614955+0000","last_undegraded":"2024-05-08T05:53:46.614955+0000","last_fullsized":"2024-05-08T05:53:46.614955+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:20.872768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"89'8","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513689+0000","last_change":"2024-05-08T05:50:34.373044+0000","last_active":"2024-05-08T05:53:46.513689+0000","last_peered":"2024-05-08T05:53:46.513689+0000","last_clean":"2024-05-08T05:53:46.513689+0000","last_became_active":"2024-05-08T05:50:34.372580+0000","last_became_peered":"2024-05-08T05:50:34.372580+0000","last_unstale":"2024-05-08T05:53:46.513689+0000","last_undegraded":"2024-05-08T05:53:46.513689+0000","last_fullsized":"2024-05-08T05:53:46.513689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:47:51.359585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614248+0000","last_change":"2024-05-08T05:50:12.716199+0000","last_active":"2024-05-08T05:53:46.614248+0000","last_peered":"2024-05-08T05:53:46.614248+0000","last_clean":"2024-05-08T05:53:46.614248+0000","last_became_active":"2024-05-08T05:50:12.715688+0000","last_became_peered":"2024-05-08T05:50:12.715688+0000","last_unstale":"2024-05-08T05:53:46.614248+0000","last_undegraded":"2024-05-08T05:53:46.614248+0000","last_fullsized":"2024-05-08T05:53:46.614248+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:17:45.644617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509660+0000","last_change":"2024-05-08T05:49:56.603910+0000","last_active":"2024-05-08T05:53:46.509660+0000","last_peered":"2024-05-08T05:53:46.509660+0000","last_clean":"2024-05-08T05:53:46.509660+0000","last_became_active":"2024-05-08T05:49:56.603188+0000","last_became_peered":"2024-05-08T05:49:56.603188+0000","last_unstale":"2024-05-08T05:53:46.509660+0000","last_undegraded":"2024-05-08T05:53:46.509660+0000","last_fullsized":"2024-05-08T05:53:46.509660+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:32:43.130463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614291+0000","last_change":"2024-05-08T05:49:59.633881+0000","last_active":"2024-05-08T05:53:46.614291+0000","last_peered":"2024-05-08T05:53:46.614291+0000","last_clean":"2024-05-08T05:53:46.614291+0000","last_became_active":"2024-05-08T05:49:59.633553+0000","last_became_peered":"2024-05-08T05:49:59.633553+0000","last_unstale":"2024-05-08T05:53:46.614291+0000","last_undegraded":"2024-05-08T05:53:46.614291+0000","last_fullsized":"2024-05-08T05:53:46.614291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:48.718124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614723+0000","last_change":"2024-05-08T05:50:12.704907+0000","last_active":"2024-05-08T05:53:46.614723+0000","last_peered":"2024-05-08T05:53:46.614723+0000","last_clean":"2024-05-08T05:53:46.614723+0000","last_became_active":"2024-05-08T05:50:12.704501+0000","last_became_peered":"2024-05-08T05:50:12.704501+0000","last_unstale":"2024-05-08T05:53:46.614723+0000","last_undegraded":"2024-05-08T05:53:46.614723+0000","last_fullsized":"2024-05-08T05:53:46.614723+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:56.214919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513070+0000","last_change":"2024-05-08T05:49:56.613471+0000","last_active":"2024-05-08T05:53:46.513070+0000","last_peered":"2024-05-08T05:53:46.513070+0000","last_clean":"2024-05-08T05:53:46.513070+0000","last_became_active":"2024-05-08T05:49:56.613085+0000","last_became_peered":"2024-05-08T05:49:56.613085+0000","last_unstale":"2024-05-08T05:53:46.513070+0000","last_undegraded":"2024-05-08T05:53:46.513070+0000","last_fullsized":"2024-05-08T05:53:46.513070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:15:19.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"89'11","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616751+0000","last_change":"2024-05-08T05:49:59.638158+0000","last_active":"2024-05-08T05:53:46.616751+0000","last_peered":"2024-05-08T05:53:46.616751+0000","last_clean":"2024-05-08T05:53:46.616751+0000","last_became_active":"2024-05-08T05:49:59.637832+0000","last_became_peered":"2024-05-08T05:49:59.637832+0000","last_unstale":"2024-05-08T05:53:46.616751+0000","last_undegraded":"2024-05-08T05:53:46.616751+0000","last_fullsized":"2024-05-08T05:53:46.616751+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:02:12.097123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614639+0000","last_change":"2024-05-08T05:50:12.708137+0000","last_active":"2024-05-08T05:53:46.614639+0000","last_peered":"2024-05-08T05:53:46.614639+0000","last_clean":"2024-05-08T05:53:46.614639+0000","last_became_active":"2024-05-08T05:50:12.707738+0000","last_became_peered":"2024-05-08T05:50:12.707738+0000","last_unstale":"2024-05-08T05:53:46.614639+0000","last_undegraded":"2024-05-08T05:53:46.614639+0000","last_fullsized":"2024-05-08T05:53:46.614639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:58.117743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615870+0000","last_change":"2024-05-08T05:49:56.611664+0000","last_active":"2024-05-08T05:53:46.615870+0000","last_peered":"2024-05-08T05:53:46.615870+0000","last_clean":"2024-05-08T05:53:46.615870+0000","last_became_active":"2024-05-08T05:49:56.611292+0000","last_became_peered":"2024-05-08T05:49:56.611292+0000","last_unstale":"2024-05-08T05:53:46.615870+0000","last_undegraded":"2024-05-08T05:53:46.615870+0000","last_fullsized":"2024-05-08T05:53:46.615870+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:38:29.874246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615261+0000","last_change":"2024-05-08T05:49:59.651736+0000","last_active":"2024-05-08T05:53:46.615261+0000","last_peered":"2024-05-08T05:53:46.615261+0000","last_clean":"2024-05-08T05:53:46.615261+0000","last_became_active":"2024-05-08T05:49:59.651404+0000","last_became_peered":"2024-05-08T05:49:59.651404+0000","last_unstale":"2024-05-08T05:53:46.615261+0000","last_undegraded":"2024-05-08T05:53:46.615261+0000","last_fullsized":"2024-05-08T05:53:46.615261+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:36:47.909783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616532+0000","last_change":"2024-05-08T05:50:12.712547+0000","last_active":"2024-05-08T05:53:46.616532+0000","last_peered":"2024-05-08T05:53:46.616532+0000","last_clean":"2024-05-08T05:53:46.616532+0000","last_became_active":"2024-05-08T05:50:12.712153+0000","last_became_peered":"2024-05-08T05:50:12.712153+0000","last_unstale":"2024-05-08T05:53:46.616532+0000","last_undegraded":"2024-05-08T05:53:46.616532+0000","last_fullsized":"2024-05-08T05:53:46.616532+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:44:30.738634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511819+0000","last_change":"2024-05-08T05:49:56.614672+0000","last_active":"2024-05-08T05:53:46.511819+0000","last_peered":"2024-05-08T05:53:46.511819+0000","last_clean":"2024-05-08T05:53:46.511819+0000","last_became_active":"2024-05-08T05:49:56.614324+0000","last_became_peered":"2024-05-08T05:49:56.614324+0000","last_unstale":"2024-05-08T05:53:46.511819+0000","last_undegraded":"2024-05-08T05:53:46.511819+0000","last_fullsized":"2024-05-08T05:53:46.511819+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:31.853587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614831+0000","last_change":"2024-05-08T05:49:59.642744+0000","last_active":"2024-05-08T05:53:46.614831+0000","last_peered":"2024-05-08T05:53:46.614831+0000","last_clean":"2024-05-08T05:53:46.614831+0000","last_became_active":"2024-05-08T05:49:59.642351+0000","last_became_peered":"2024-05-08T05:49:59.642351+0000","last_unstale":"2024-05-08T05:53:46.614831+0000","last_undegraded":"2024-05-08T05:53:46.614831+0000","last_fullsized":"2024-05-08T05:53:46.614831+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:46:27.252403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614222+0000","last_change":"2024-05-08T05:50:12.712522+0000","last_active":"2024-05-08T05:53:46.614222+0000","last_peered":"2024-05-08T05:53:46.614222+0000","last_clean":"2024-05-08T05:53:46.614222+0000","last_became_active":"2024-05-08T05:50:12.712005+0000","last_became_peered":"2024-05-08T05:50:12.712005+0000","last_unstale":"2024-05-08T05:53:46.614222+0000","last_undegraded":"2024-05-08T05:53:46.614222+0000","last_fullsized":"2024-05-08T05:53:46.614222+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:25.556337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"78'3","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513588+0000","last_change":"2024-05-08T05:49:56.626998+0000","last_active":"2024-05-08T05:53:46.513588+0000","last_peered":"2024-05-08T05:53:46.513588+0000","last_clean":"2024-05-08T05:53:46.513588+0000","last_became_active":"2024-05-08T05:49:56.626552+0000","last_became_peered":"2024-05-08T05:49:56.626552+0000","last_unstale":"2024-05-08T05:53:46.513588+0000","last_undegraded":"2024-05-08T05:53:46.513588+0000","last_fullsized":"2024-05-08T05:53:46.513588+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:27:13.254255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614838+0000","last_change":"2024-05-08T05:49:59.640424+0000","last_active":"2024-05-08T05:53:46.614838+0000","last_peered":"2024-05-08T05:53:46.614838+0000","last_clean":"2024-05-08T05:53:46.614838+0000","last_became_active":"2024-05-08T05:49:59.639526+0000","last_became_peered":"2024-05-08T05:49:59.639526+0000","last_unstale":"2024-05-08T05:53:46.614838+0000","last_undegraded":"2024-05-08T05:53:46.614838+0000","last_fullsized":"2024-05-08T05:53:46.614838+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:40:34.604616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614704+0000","last_change":"2024-05-08T05:50:34.308283+0000","last_active":"2024-05-08T05:53:46.614704+0000","last_peered":"2024-05-08T05:53:46.614704+0000","last_clean":"2024-05-08T05:53:46.614704+0000","last_became_active":"2024-05-08T05:50:34.307810+0000","last_became_peered":"2024-05-08T05:50:34.307810+0000","last_unstale":"2024-05-08T05:53:46.614704+0000","last_undegraded":"2024-05-08T05:53:46.614704+0000","last_fullsized":"2024-05-08T05:53:46.614704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:18.074855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'41","reported_seq":440,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:51.816305+0000","last_change":"2024-05-08T05:49:56.613434+0000","last_active":"2024-05-08T05:53:51.816305+0000","last_peered":"2024-05-08T05:53:51.816305+0000","last_clean":"2024-05-08T05:53:51.816305+0000","last_became_active":"2024-05-08T05:49:56.612964+0000","last_became_peered":"2024-05-08T05:49:56.612964+0000","last_unstale":"2024-05-08T05:53:51.816305+0000","last_undegraded":"2024-05-08T05:53:51.816305+0000","last_fullsized":"2024-05-08T05:53:51.816305+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:49.498798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71839,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":352,"num_read_kb":246,"num_write":41,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614614+0000","last_change":"2024-05-08T05:49:59.641026+0000","last_active":"2024-05-08T05:53:46.614614+0000","last_peered":"2024-05-08T05:53:46.614614+0000","last_clean":"2024-05-08T05:53:46.614614+0000","last_became_active":"2024-05-08T05:49:59.640695+0000","last_became_peered":"2024-05-08T05:49:59.640695+0000","last_unstale":"2024-05-08T05:53:46.614614+0000","last_undegraded":"2024-05-08T05:53:46.614614+0000","last_fullsized":"2024-05-08T05:53:46.614614+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:44.387933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513963+0000","last_change":"2024-05-08T05:50:12.705974+0000","last_active":"2024-05-08T05:53:46.513963+0000","last_peered":"2024-05-08T05:53:46.513963+0000","last_clean":"2024-05-08T05:53:46.513963+0000","last_became_active":"2024-05-08T05:50:12.705588+0000","last_became_peered":"2024-05-08T05:50:12.705588+0000","last_unstale":"2024-05-08T05:53:46.513963+0000","last_undegraded":"2024-05-08T05:53:46.513963+0000","last_fullsized":"2024-05-08T05:53:46.513963+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:42.863939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614240+0000","last_change":"2024-05-08T05:49:56.613344+0000","last_active":"2024-05-08T05:53:46.614240+0000","last_peered":"2024-05-08T05:53:46.614240+0000","last_clean":"2024-05-08T05:53:46.614240+0000","last_became_active":"2024-05-08T05:49:56.612928+0000","last_became_peered":"2024-05-08T05:49:56.612928+0000","last_unstale":"2024-05-08T05:53:46.614240+0000","last_undegraded":"2024-05-08T05:53:46.614240+0000","last_fullsized":"2024-05-08T05:53:46.614240+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:41:11.246525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509653+0000","last_change":"2024-05-08T05:49:59.646540+0000","last_active":"2024-05-08T05:53:46.509653+0000","last_peered":"2024-05-08T05:53:46.509653+0000","last_clean":"2024-05-08T05:53:46.509653+0000","last_became_active":"2024-05-08T05:49:59.646237+0000","last_became_peered":"2024-05-08T05:49:59.646237+0000","last_unstale":"2024-05-08T05:53:46.509653+0000","last_undegraded":"2024-05-08T05:53:46.509653+0000","last_fullsized":"2024-05-08T05:53:46.509653+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:32.695058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615055+0000","last_change":"2024-05-08T05:50:12.712192+0000","last_active":"2024-05-08T05:53:46.615055+0000","last_peered":"2024-05-08T05:53:46.615055+0000","last_clean":"2024-05-08T05:53:46.615055+0000","last_became_active":"2024-05-08T05:50:12.711735+0000","last_became_peered":"2024-05-08T05:50:12.711735+0000","last_unstale":"2024-05-08T05:53:46.615055+0000","last_undegraded":"2024-05-08T05:53:46.615055+0000","last_fullsized":"2024-05-08T05:53:46.615055+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:59.461599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513354+0000","last_change":"2024-05-08T05:49:56.610174+0000","last_active":"2024-05-08T05:53:46.513354+0000","last_peered":"2024-05-08T05:53:46.513354+0000","last_clean":"2024-05-08T05:53:46.513354+0000","last_became_active":"2024-05-08T05:49:56.609736+0000","last_became_peered":"2024-05-08T05:49:56.609736+0000","last_unstale":"2024-05-08T05:53:46.513354+0000","last_undegraded":"2024-05-08T05:53:46.513354+0000","last_fullsized":"2024-05-08T05:53:46.513354+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:31.503951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"89'15","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511440+0000","last_change":"2024-05-08T05:49:59.651811+0000","last_active":"2024-05-08T05:53:46.511440+0000","last_peered":"2024-05-08T05:53:46.511440+0000","last_clean":"2024-05-08T05:53:46.511440+0000","last_became_active":"2024-05-08T05:49:59.651181+0000","last_became_peered":"2024-05-08T05:49:59.651181+0000","last_unstale":"2024-05-08T05:53:46.511440+0000","last_undegraded":"2024-05-08T05:53:46.511440+0000","last_fullsized":"2024-05-08T05:53:46.511440+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:51:27.515612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617224+0000","last_change":"2024-05-08T05:50:12.715995+0000","last_active":"2024-05-08T05:53:46.617224+0000","last_peered":"2024-05-08T05:53:46.617224+0000","last_clean":"2024-05-08T05:53:46.617224+0000","last_became_active":"2024-05-08T05:50:12.715613+0000","last_became_peered":"2024-05-08T05:50:12.715613+0000","last_unstale":"2024-05-08T05:53:46.617224+0000","last_undegraded":"2024-05-08T05:53:46.617224+0000","last_fullsized":"2024-05-08T05:53:46.617224+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:51.094889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613969+0000","last_change":"2024-05-08T05:49:56.611868+0000","last_active":"2024-05-08T05:53:46.613969+0000","last_peered":"2024-05-08T05:53:46.613969+0000","last_clean":"2024-05-08T05:53:46.613969+0000","last_became_active":"2024-05-08T05:49:56.611186+0000","last_became_peered":"2024-05-08T05:49:56.611186+0000","last_unstale":"2024-05-08T05:53:46.613969+0000","last_undegraded":"2024-05-08T05:53:46.613969+0000","last_fullsized":"2024-05-08T05:53:46.613969+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:22.959035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511462+0000","last_change":"2024-05-08T05:49:59.643449+0000","last_active":"2024-05-08T05:53:46.511462+0000","last_peered":"2024-05-08T05:53:46.511462+0000","last_clean":"2024-05-08T05:53:46.511462+0000","last_became_active":"2024-05-08T05:49:59.643088+0000","last_became_peered":"2024-05-08T05:49:59.643088+0000","last_unstale":"2024-05-08T05:53:46.511462+0000","last_undegraded":"2024-05-08T05:53:46.511462+0000","last_fullsized":"2024-05-08T05:53:46.511462+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:43:10.055357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510363+0000","last_change":"2024-05-08T05:50:12.719734+0000","last_active":"2024-05-08T05:53:46.510363+0000","last_peered":"2024-05-08T05:53:46.510363+0000","last_clean":"2024-05-08T05:53:46.510363+0000","last_became_active":"2024-05-08T05:50:12.719279+0000","last_became_peered":"2024-05-08T05:50:12.719279+0000","last_unstale":"2024-05-08T05:53:46.510363+0000","last_undegraded":"2024-05-08T05:53:46.510363+0000","last_fullsized":"2024-05-08T05:53:46.510363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:54:59.620365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"78'8","reported_seq":71,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614616+0000","last_change":"2024-05-08T05:50:34.315867+0000","last_active":"2024-05-08T05:53:46.614616+0000","last_peered":"2024-05-08T05:53:46.614616+0000","last_clean":"2024-05-08T05:53:46.614616+0000","last_became_active":"2024-05-08T05:50:34.305002+0000","last_became_peered":"2024-05-08T05:50:34.305002+0000","last_unstale":"2024-05-08T05:53:46.614616+0000","last_undegraded":"2024-05-08T05:53:46.614616+0000","last_fullsized":"2024-05-08T05:53:46.614616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:07.825901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513265+0000","last_change":"2024-05-08T05:49:59.647148+0000","last_active":"2024-05-08T05:53:46.513265+0000","last_peered":"2024-05-08T05:53:46.513265+0000","last_clean":"2024-05-08T05:53:46.513265+0000","last_became_active":"2024-05-08T05:49:59.646562+0000","last_became_peered":"2024-05-08T05:49:59.646562+0000","last_unstale":"2024-05-08T05:53:46.513265+0000","last_undegraded":"2024-05-08T05:53:46.513265+0000","last_fullsized":"2024-05-08T05:53:46.513265+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:41:09.939902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513186+0000","last_change":"2024-05-08T05:50:12.706406+0000","last_active":"2024-05-08T05:53:46.513186+0000","last_peered":"2024-05-08T05:53:46.513186+0000","last_clean":"2024-05-08T05:53:46.513186+0000","last_became_active":"2024-05-08T05:50:12.705956+0000","last_became_peered":"2024-05-08T05:50:12.705956+0000","last_unstale":"2024-05-08T05:53:46.513186+0000","last_undegraded":"2024-05-08T05:53:46.513186+0000","last_fullsized":"2024-05-08T05:53:46.513186+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:18:39.844257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"78'1","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615070+0000","last_change":"2024-05-08T05:49:56.611462+0000","last_active":"2024-05-08T05:53:46.615070+0000","last_peered":"2024-05-08T05:53:46.615070+0000","last_clean":"2024-05-08T05:53:46.615070+0000","last_became_active":"2024-05-08T05:49:56.611011+0000","last_became_peered":"2024-05-08T05:49:56.611011+0000","last_unstale":"2024-05-08T05:53:46.615070+0000","last_undegraded":"2024-05-08T05:53:46.615070+0000","last_fullsized":"2024-05-08T05:53:46.615070+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:58.541004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"89'5","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614749+0000","last_change":"2024-05-08T05:50:34.310426+0000","last_active":"2024-05-08T05:53:46.614749+0000","last_peered":"2024-05-08T05:53:46.614749+0000","last_clean":"2024-05-08T05:53:46.614749+0000","last_became_active":"2024-05-08T05:50:34.309986+0000","last_became_peered":"2024-05-08T05:50:34.309986+0000","last_unstale":"2024-05-08T05:53:46.614749+0000","last_undegraded":"2024-05-08T05:53:46.614749+0000","last_fullsized":"2024-05-08T05:53:46.614749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:27:25.366314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614135+0000","last_change":"2024-05-08T05:50:34.309753+0000","last_active":"2024-05-08T05:53:46.614135+0000","last_peered":"2024-05-08T05:53:46.614135+0000","last_clean":"2024-05-08T05:53:46.614135+0000","last_became_active":"2024-05-08T05:50:34.309338+0000","last_became_peered":"2024-05-08T05:50:34.309338+0000","last_unstale":"2024-05-08T05:53:46.614135+0000","last_undegraded":"2024-05-08T05:53:46.614135+0000","last_fullsized":"2024-05-08T05:53:46.614135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:43.167100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617130+0000","last_change":"2024-05-08T05:50:34.312120+0000","last_active":"2024-05-08T05:53:46.617130+0000","last_peered":"2024-05-08T05:53:46.617130+0000","last_clean":"2024-05-08T05:53:46.617130+0000","last_became_active":"2024-05-08T05:50:34.311650+0000","last_became_peered":"2024-05-08T05:50:34.311650+0000","last_unstale":"2024-05-08T05:53:46.617130+0000","last_undegraded":"2024-05-08T05:53:46.617130+0000","last_fullsized":"2024-05-08T05:53:46.617130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:28:57.984724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'4","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615364+0000","last_change":"2024-05-08T05:49:59.648911+0000","last_active":"2024-05-08T05:53:46.615364+0000","last_peered":"2024-05-08T05:53:46.615364+0000","last_clean":"2024-05-08T05:53:46.615364+0000","last_became_active":"2024-05-08T05:49:59.648560+0000","last_became_peered":"2024-05-08T05:49:59.648560+0000","last_unstale":"2024-05-08T05:53:46.615364+0000","last_undegraded":"2024-05-08T05:53:46.615364+0000","last_fullsized":"2024-05-08T05:53:46.615364+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:44.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510122+0000","last_change":"2024-05-08T05:50:12.703694+0000","last_active":"2024-05-08T05:53:46.510122+0000","last_peered":"2024-05-08T05:53:46.510122+0000","last_clean":"2024-05-08T05:53:46.510122+0000","last_became_active":"2024-05-08T05:50:12.702799+0000","last_became_peered":"2024-05-08T05:50:12.702799+0000","last_unstale":"2024-05-08T05:53:46.510122+0000","last_undegraded":"2024-05-08T05:53:46.510122+0000","last_fullsized":"2024-05-08T05:53:46.510122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:40.407699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511385+0000","last_change":"2024-05-08T05:49:56.614776+0000","last_active":"2024-05-08T05:53:46.511385+0000","last_peered":"2024-05-08T05:53:46.511385+0000","last_clean":"2024-05-08T05:53:46.511385+0000","last_became_active":"2024-05-08T05:49:56.614299+0000","last_became_peered":"2024-05-08T05:49:56.614299+0000","last_unstale":"2024-05-08T05:53:46.511385+0000","last_undegraded":"2024-05-08T05:53:46.511385+0000","last_fullsized":"2024-05-08T05:53:46.511385+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:18:42.677557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"89'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615576+0000","last_change":"2024-05-08T05:50:34.306858+0000","last_active":"2024-05-08T05:53:46.615576+0000","last_peered":"2024-05-08T05:53:46.615576+0000","last_clean":"2024-05-08T05:53:46.615576+0000","last_became_active":"2024-05-08T05:50:34.306459+0000","last_became_peered":"2024-05-08T05:50:34.306459+0000","last_unstale":"2024-05-08T05:53:46.615576+0000","last_undegraded":"2024-05-08T05:53:46.615576+0000","last_fullsized":"2024-05-08T05:53:46.615576+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:14.787886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510472+0000","last_change":"2024-05-08T05:50:12.706616+0000","last_active":"2024-05-08T05:53:46.510472+0000","last_peered":"2024-05-08T05:53:46.510472+0000","last_clean":"2024-05-08T05:53:46.510472+0000","last_became_active":"2024-05-08T05:50:12.705502+0000","last_became_peered":"2024-05-08T05:50:12.705502+0000","last_unstale":"2024-05-08T05:53:46.510472+0000","last_undegraded":"2024-05-08T05:53:46.510472+0000","last_fullsized":"2024-05-08T05:53:46.510472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:51:53.644848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513347+0000","last_change":"2024-05-08T05:49:59.648739+0000","last_active":"2024-05-08T05:53:46.513347+0000","last_peered":"2024-05-08T05:53:46.513347+0000","last_clean":"2024-05-08T05:53:46.513347+0000","last_became_active":"2024-05-08T05:49:59.648250+0000","last_became_peered":"2024-05-08T05:49:59.648250+0000","last_unstale":"2024-05-08T05:53:46.513347+0000","last_undegraded":"2024-05-08T05:53:46.513347+0000","last_fullsized":"2024-05-08T05:53:46.513347+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:05:57.504262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509535+0000","last_change":"2024-05-08T05:49:56.626149+0000","last_active":"2024-05-08T05:53:46.509535+0000","last_peered":"2024-05-08T05:53:46.509535+0000","last_clean":"2024-05-08T05:53:46.509535+0000","last_became_active":"2024-05-08T05:49:56.625830+0000","last_became_peered":"2024-05-08T05:49:56.625830+0000","last_unstale":"2024-05-08T05:53:46.509535+0000","last_undegraded":"2024-05-08T05:53:46.509535+0000","last_fullsized":"2024-05-08T05:53:46.509535+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:04.430483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511414+0000","last_change":"2024-05-08T05:50:12.710096+0000","last_active":"2024-05-08T05:53:46.511414+0000","last_peered":"2024-05-08T05:53:46.511414+0000","last_clean":"2024-05-08T05:53:46.511414+0000","last_became_active":"2024-05-08T05:50:12.709645+0000","last_became_peered":"2024-05-08T05:50:12.709645+0000","last_unstale":"2024-05-08T05:53:46.511414+0000","last_undegraded":"2024-05-08T05:53:46.511414+0000","last_fullsized":"2024-05-08T05:53:46.511414+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:52.016791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615039+0000","last_change":"2024-05-08T05:49:59.637067+0000","last_active":"2024-05-08T05:53:46.615039+0000","last_peered":"2024-05-08T05:53:46.615039+0000","last_clean":"2024-05-08T05:53:46.615039+0000","last_became_active":"2024-05-08T05:49:59.636753+0000","last_became_peered":"2024-05-08T05:49:59.636753+0000","last_unstale":"2024-05-08T05:53:46.615039+0000","last_undegraded":"2024-05-08T05:53:46.615039+0000","last_fullsized":"2024-05-08T05:53:46.615039+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:02:39.470623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614261+0000","last_change":"2024-05-08T05:49:56.614748+0000","last_active":"2024-05-08T05:53:46.614261+0000","last_peered":"2024-05-08T05:53:46.614261+0000","last_clean":"2024-05-08T05:53:46.614261+0000","last_became_active":"2024-05-08T05:49:56.614349+0000","last_became_peered":"2024-05-08T05:49:56.614349+0000","last_unstale":"2024-05-08T05:53:46.614261+0000","last_undegraded":"2024-05-08T05:53:46.614261+0000","last_fullsized":"2024-05-08T05:53:46.614261+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:51.013871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"90'99","reported_seq":150,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:48.610665+0000","last_change":"2024-05-08T05:48:30.268846+0000","last_active":"2024-05-08T05:53:48.610665+0000","last_peered":"2024-05-08T05:53:48.610665+0000","last_clean":"2024-05-08T05:53:48.610665+0000","last_became_active":"2024-05-08T05:48:30.264509+0000","last_became_peered":"2024-05-08T05:48:30.264509+0000","last_unstale":"2024-05-08T05:53:48.610665+0000","last_undegraded":"2024-05-08T05:53:48.610665+0000","last_fullsized":"2024-05-08T05:53:48.610665+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:45:02.196749+0000","last_clean_scrub_stamp":"2024-05-08T05:45:02.196749+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:48.545190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513641+0000","last_change":"2024-05-08T05:50:12.719989+0000","last_active":"2024-05-08T05:53:46.513641+0000","last_peered":"2024-05-08T05:53:46.513641+0000","last_clean":"2024-05-08T05:53:46.513641+0000","last_became_active":"2024-05-08T05:50:12.719442+0000","last_became_peered":"2024-05-08T05:50:12.719442+0000","last_unstale":"2024-05-08T05:53:46.513641+0000","last_undegraded":"2024-05-08T05:53:46.513641+0000","last_fullsized":"2024-05-08T05:53:46.513641+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:03.521053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"89'5","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509529+0000","last_change":"2024-05-08T05:49:59.649690+0000","last_active":"2024-05-08T05:53:46.509529+0000","last_peered":"2024-05-08T05:53:46.509529+0000","last_clean":"2024-05-08T05:53:46.509529+0000","last_became_active":"2024-05-08T05:49:59.649368+0000","last_became_peered":"2024-05-08T05:49:59.649368+0000","last_unstale":"2024-05-08T05:53:46.509529+0000","last_undegraded":"2024-05-08T05:53:46.509529+0000","last_fullsized":"2024-05-08T05:53:46.509529+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:58.900274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511399+0000","last_change":"2024-05-08T05:49:56.613522+0000","last_active":"2024-05-08T05:53:46.511399+0000","last_peered":"2024-05-08T05:53:46.511399+0000","last_clean":"2024-05-08T05:53:46.511399+0000","last_became_active":"2024-05-08T05:49:56.613101+0000","last_became_peered":"2024-05-08T05:49:56.613101+0000","last_unstale":"2024-05-08T05:53:46.511399+0000","last_undegraded":"2024-05-08T05:53:46.511399+0000","last_fullsized":"2024-05-08T05:53:46.511399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:55:24.409936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614234+0000","last_change":"2024-05-08T05:50:12.705000+0000","last_active":"2024-05-08T05:53:46.614234+0000","last_peered":"2024-05-08T05:53:46.614234+0000","last_clean":"2024-05-08T05:53:46.614234+0000","last_became_active":"2024-05-08T05:50:12.704515+0000","last_became_peered":"2024-05-08T05:50:12.704515+0000","last_unstale":"2024-05-08T05:53:46.614234+0000","last_undegraded":"2024-05-08T05:53:46.614234+0000","last_fullsized":"2024-05-08T05:53:46.614234+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:44.076292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614508+0000","last_change":"2024-05-08T05:49:56.626751+0000","last_active":"2024-05-08T05:53:46.614508+0000","last_peered":"2024-05-08T05:53:46.614508+0000","last_clean":"2024-05-08T05:53:46.614508+0000","last_became_active":"2024-05-08T05:49:56.626380+0000","last_became_peered":"2024-05-08T05:49:56.626380+0000","last_unstale":"2024-05-08T05:53:46.614508+0000","last_undegraded":"2024-05-08T05:53:46.614508+0000","last_fullsized":"2024-05-08T05:53:46.614508+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:16.476614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"89'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510235+0000","last_change":"2024-05-08T05:49:59.650209+0000","last_active":"2024-05-08T05:53:46.510235+0000","last_peered":"2024-05-08T05:53:46.510235+0000","last_clean":"2024-05-08T05:53:46.510235+0000","last_became_active":"2024-05-08T05:49:59.649349+0000","last_became_peered":"2024-05-08T05:49:59.649349+0000","last_unstale":"2024-05-08T05:53:46.510235+0000","last_undegraded":"2024-05-08T05:53:46.510235+0000","last_fullsized":"2024-05-08T05:53:46.510235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:35:22.931177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614195+0000","last_change":"2024-05-08T05:50:12.702229+0000","last_active":"2024-05-08T05:53:46.614195+0000","last_peered":"2024-05-08T05:53:46.614195+0000","last_clean":"2024-05-08T05:53:46.614195+0000","last_became_active":"2024-05-08T05:50:12.701817+0000","last_became_peered":"2024-05-08T05:50:12.701817+0000","last_unstale":"2024-05-08T05:53:46.614195+0000","last_undegraded":"2024-05-08T05:53:46.614195+0000","last_fullsized":"2024-05-08T05:53:46.614195+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:24:45.619310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'100","reported_seq":239,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:48.737785+0000","last_change":"2024-05-08T05:49:56.605635+0000","last_active":"2024-05-08T05:53:48.737785+0000","last_peered":"2024-05-08T05:53:48.737785+0000","last_clean":"2024-05-08T05:53:48.737785+0000","last_became_active":"2024-05-08T05:49:56.605265+0000","last_became_peered":"2024-05-08T05:49:56.605265+0000","last_unstale":"2024-05-08T05:53:48.737785+0000","last_undegraded":"2024-05-08T05:53:48.737785+0000","last_fullsized":"2024-05-08T05:53:48.737785+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:35.455985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":184,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615918+0000","last_change":"2024-05-08T05:49:59.633289+0000","last_active":"2024-05-08T05:53:46.615918+0000","last_peered":"2024-05-08T05:53:46.615918+0000","last_clean":"2024-05-08T05:53:46.615918+0000","last_became_active":"2024-05-08T05:49:59.632810+0000","last_became_peered":"2024-05-08T05:49:59.632810+0000","last_unstale":"2024-05-08T05:53:46.615918+0000","last_undegraded":"2024-05-08T05:53:46.615918+0000","last_fullsized":"2024-05-08T05:53:46.615918+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:40.015381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613810+0000","last_change":"2024-05-08T05:50:12.705018+0000","last_active":"2024-05-08T05:53:46.613810+0000","last_peered":"2024-05-08T05:53:46.613810+0000","last_clean":"2024-05-08T05:53:46.613810+0000","last_became_active":"2024-05-08T05:50:12.704587+0000","last_became_peered":"2024-05-08T05:50:12.704587+0000","last_unstale":"2024-05-08T05:53:46.613810+0000","last_undegraded":"2024-05-08T05:53:46.613810+0000","last_fullsized":"2024-05-08T05:53:46.613810+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:05:21.186438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"78'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614825+0000","last_change":"2024-05-08T05:49:56.843346+0000","last_active":"2024-05-08T05:53:46.614825+0000","last_peered":"2024-05-08T05:53:46.614825+0000","last_clean":"2024-05-08T05:53:46.614825+0000","last_became_active":"2024-05-08T05:49:56.843070+0000","last_became_peered":"2024-05-08T05:49:56.843070+0000","last_unstale":"2024-05-08T05:53:46.614825+0000","last_undegraded":"2024-05-08T05:53:46.614825+0000","last_fullsized":"2024-05-08T05:53:46.614825+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:50:37.127468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614833+0000","last_change":"2024-05-08T05:49:59.642469+0000","last_active":"2024-05-08T05:53:46.614833+0000","last_peered":"2024-05-08T05:53:46.614833+0000","last_clean":"2024-05-08T05:53:46.614833+0000","last_became_active":"2024-05-08T05:49:59.642119+0000","last_became_peered":"2024-05-08T05:49:59.642119+0000","last_unstale":"2024-05-08T05:53:46.614833+0000","last_undegraded":"2024-05-08T05:53:46.614833+0000","last_fullsized":"2024-05-08T05:53:46.614833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:40:33.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614454+0000","last_change":"2024-05-08T05:50:12.720153+0000","last_active":"2024-05-08T05:53:46.614454+0000","last_peered":"2024-05-08T05:53:46.614454+0000","last_clean":"2024-05-08T05:53:46.614454+0000","last_became_active":"2024-05-08T05:50:12.719375+0000","last_became_peered":"2024-05-08T05:50:12.719375+0000","last_unstale":"2024-05-08T05:53:46.614454+0000","last_undegraded":"2024-05-08T05:53:46.614454+0000","last_fullsized":"2024-05-08T05:53:46.614454+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:08.880305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613822+0000","last_change":"2024-05-08T05:49:56.626839+0000","last_active":"2024-05-08T05:53:46.613822+0000","last_peered":"2024-05-08T05:53:46.613822+0000","last_clean":"2024-05-08T05:53:46.613822+0000","last_became_active":"2024-05-08T05:49:56.626568+0000","last_became_peered":"2024-05-08T05:49:56.626568+0000","last_unstale":"2024-05-08T05:53:46.613822+0000","last_undegraded":"2024-05-08T05:53:46.613822+0000","last_fullsized":"2024-05-08T05:53:46.613822+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:22.120598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"89'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613910+0000","last_change":"2024-05-08T05:49:59.643016+0000","last_active":"2024-05-08T05:53:46.613910+0000","last_peered":"2024-05-08T05:53:46.613910+0000","last_clean":"2024-05-08T05:53:46.613910+0000","last_became_active":"2024-05-08T05:49:59.642677+0000","last_became_peered":"2024-05-08T05:49:59.642677+0000","last_unstale":"2024-05-08T05:53:46.613910+0000","last_undegraded":"2024-05-08T05:53:46.613910+0000","last_fullsized":"2024-05-08T05:53:46.613910+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:35.488591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.617785+0000","last_change":"2024-05-08T05:50:12.718889+0000","last_active":"2024-05-08T05:53:46.617785+0000","last_peered":"2024-05-08T05:53:46.617785+0000","last_clean":"2024-05-08T05:53:46.617785+0000","last_became_active":"2024-05-08T05:50:12.718447+0000","last_became_peered":"2024-05-08T05:50:12.718447+0000","last_unstale":"2024-05-08T05:53:46.617785+0000","last_undegraded":"2024-05-08T05:53:46.617785+0000","last_fullsized":"2024-05-08T05:53:46.617785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:08:32.465469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614741+0000","last_change":"2024-05-08T05:49:56.611337+0000","last_active":"2024-05-08T05:53:46.614741+0000","last_peered":"2024-05-08T05:53:46.614741+0000","last_clean":"2024-05-08T05:53:46.614741+0000","last_became_active":"2024-05-08T05:49:56.610878+0000","last_became_peered":"2024-05-08T05:49:56.610878+0000","last_unstale":"2024-05-08T05:53:46.614741+0000","last_undegraded":"2024-05-08T05:53:46.614741+0000","last_fullsized":"2024-05-08T05:53:46.614741+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.834216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615035+0000","last_change":"2024-05-08T05:49:59.634841+0000","last_active":"2024-05-08T05:53:46.615035+0000","last_peered":"2024-05-08T05:53:46.615035+0000","last_clean":"2024-05-08T05:53:46.615035+0000","last_became_active":"2024-05-08T05:49:59.634364+0000","last_became_peered":"2024-05-08T05:49:59.634364+0000","last_unstale":"2024-05-08T05:53:46.615035+0000","last_undegraded":"2024-05-08T05:53:46.615035+0000","last_fullsized":"2024-05-08T05:53:46.615035+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:13.046041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614455+0000","last_change":"2024-05-08T05:50:12.719789+0000","last_active":"2024-05-08T05:53:46.614455+0000","last_peered":"2024-05-08T05:53:46.614455+0000","last_clean":"2024-05-08T05:53:46.614455+0000","last_became_active":"2024-05-08T05:50:12.719286+0000","last_became_peered":"2024-05-08T05:50:12.719286+0000","last_unstale":"2024-05-08T05:53:46.614455+0000","last_undegraded":"2024-05-08T05:53:46.614455+0000","last_fullsized":"2024-05-08T05:53:46.614455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:03:20.818121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614015+0000","last_change":"2024-05-08T05:49:56.608156+0000","last_active":"2024-05-08T05:53:46.614015+0000","last_peered":"2024-05-08T05:53:46.614015+0000","last_clean":"2024-05-08T05:53:46.614015+0000","last_became_active":"2024-05-08T05:49:56.607586+0000","last_became_peered":"2024-05-08T05:49:56.607586+0000","last_unstale":"2024-05-08T05:53:46.614015+0000","last_undegraded":"2024-05-08T05:53:46.614015+0000","last_fullsized":"2024-05-08T05:53:46.614015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:42:02.740586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615562+0000","last_change":"2024-05-08T05:49:59.633017+0000","last_active":"2024-05-08T05:53:46.615562+0000","last_peered":"2024-05-08T05:53:46.615562+0000","last_clean":"2024-05-08T05:53:46.615562+0000","last_became_active":"2024-05-08T05:49:59.632665+0000","last_became_peered":"2024-05-08T05:49:59.632665+0000","last_unstale":"2024-05-08T05:53:46.615562+0000","last_undegraded":"2024-05-08T05:53:46.615562+0000","last_fullsized":"2024-05-08T05:53:46.615562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:07.865872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614622+0000","last_change":"2024-05-08T05:50:12.706327+0000","last_active":"2024-05-08T05:53:46.614622+0000","last_peered":"2024-05-08T05:53:46.614622+0000","last_clean":"2024-05-08T05:53:46.614622+0000","last_became_active":"2024-05-08T05:50:12.705595+0000","last_became_peered":"2024-05-08T05:50:12.705595+0000","last_unstale":"2024-05-08T05:53:46.614622+0000","last_undegraded":"2024-05-08T05:53:46.614622+0000","last_fullsized":"2024-05-08T05:53:46.614622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:02:33.668900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"78'2","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613925+0000","last_change":"2024-05-08T05:49:56.615863+0000","last_active":"2024-05-08T05:53:46.613925+0000","last_peered":"2024-05-08T05:53:46.613925+0000","last_clean":"2024-05-08T05:53:46.613925+0000","last_became_active":"2024-05-08T05:49:56.615506+0000","last_became_peered":"2024-05-08T05:49:56.615506+0000","last_unstale":"2024-05-08T05:53:46.613925+0000","last_undegraded":"2024-05-08T05:53:46.613925+0000","last_fullsized":"2024-05-08T05:53:46.613925+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:07.997260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"89'5","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511863+0000","last_change":"2024-05-08T05:49:59.646516+0000","last_active":"2024-05-08T05:53:46.511863+0000","last_peered":"2024-05-08T05:53:46.511863+0000","last_clean":"2024-05-08T05:53:46.511863+0000","last_became_active":"2024-05-08T05:49:59.645816+0000","last_became_peered":"2024-05-08T05:49:59.645816+0000","last_unstale":"2024-05-08T05:53:46.511863+0000","last_undegraded":"2024-05-08T05:53:46.511863+0000","last_fullsized":"2024-05-08T05:53:46.511863+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:02.069059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614904+0000","last_change":"2024-05-08T05:50:12.706177+0000","last_active":"2024-05-08T05:53:46.614904+0000","last_peered":"2024-05-08T05:53:46.614904+0000","last_clean":"2024-05-08T05:53:46.614904+0000","last_became_active":"2024-05-08T05:50:12.705706+0000","last_became_peered":"2024-05-08T05:50:12.705706+0000","last_unstale":"2024-05-08T05:53:46.614904+0000","last_undegraded":"2024-05-08T05:53:46.614904+0000","last_fullsized":"2024-05-08T05:53:46.614904+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:53.824338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614702+0000","last_change":"2024-05-08T05:49:56.843342+0000","last_active":"2024-05-08T05:53:46.614702+0000","last_peered":"2024-05-08T05:53:46.614702+0000","last_clean":"2024-05-08T05:53:46.614702+0000","last_became_active":"2024-05-08T05:49:56.842864+0000","last_became_peered":"2024-05-08T05:49:56.842864+0000","last_unstale":"2024-05-08T05:53:46.614702+0000","last_undegraded":"2024-05-08T05:53:46.614702+0000","last_fullsized":"2024-05-08T05:53:46.614702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:02.875071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511809+0000","last_change":"2024-05-08T05:49:59.648830+0000","last_active":"2024-05-08T05:53:46.511809+0000","last_peered":"2024-05-08T05:53:46.511809+0000","last_clean":"2024-05-08T05:53:46.511809+0000","last_became_active":"2024-05-08T05:49:59.647714+0000","last_became_peered":"2024-05-08T05:49:59.647714+0000","last_unstale":"2024-05-08T05:53:46.511809+0000","last_undegraded":"2024-05-08T05:53:46.511809+0000","last_fullsized":"2024-05-08T05:53:46.511809+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:01.724712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614223+0000","last_change":"2024-05-08T05:50:12.710127+0000","last_active":"2024-05-08T05:53:46.614223+0000","last_peered":"2024-05-08T05:53:46.614223+0000","last_clean":"2024-05-08T05:53:46.614223+0000","last_became_active":"2024-05-08T05:50:12.709738+0000","last_became_peered":"2024-05-08T05:50:12.709738+0000","last_unstale":"2024-05-08T05:53:46.614223+0000","last_undegraded":"2024-05-08T05:53:46.614223+0000","last_fullsized":"2024-05-08T05:53:46.614223+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:21:28.330673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'65","reported_seq":143,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:48.366961+0000","last_change":"2024-05-08T05:49:56.609690+0000","last_active":"2024-05-08T05:53:48.366961+0000","last_peered":"2024-05-08T05:53:48.366961+0000","last_clean":"2024-05-08T05:53:48.366961+0000","last_became_active":"2024-05-08T05:49:56.609251+0000","last_became_peered":"2024-05-08T05:49:56.609251+0000","last_unstale":"2024-05-08T05:53:48.366961+0000","last_undegraded":"2024-05-08T05:53:48.366961+0000","last_fullsized":"2024-05-08T05:53:48.366961+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:10:50.927832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.511491+0000","last_change":"2024-05-08T05:49:59.640125+0000","last_active":"2024-05-08T05:53:46.511491+0000","last_peered":"2024-05-08T05:53:46.511491+0000","last_clean":"2024-05-08T05:53:46.511491+0000","last_became_active":"2024-05-08T05:49:59.639729+0000","last_became_peered":"2024-05-08T05:49:59.639729+0000","last_unstale":"2024-05-08T05:53:46.511491+0000","last_undegraded":"2024-05-08T05:53:46.511491+0000","last_fullsized":"2024-05-08T05:53:46.511491+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:42:22.817730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.512982+0000","last_change":"2024-05-08T05:50:12.704625+0000","last_active":"2024-05-08T05:53:46.512982+0000","last_peered":"2024-05-08T05:53:46.512982+0000","last_clean":"2024-05-08T05:53:46.512982+0000","last_became_active":"2024-05-08T05:50:12.704259+0000","last_became_peered":"2024-05-08T05:50:12.704259+0000","last_unstale":"2024-05-08T05:53:46.512982+0000","last_undegraded":"2024-05-08T05:53:46.512982+0000","last_fullsized":"2024-05-08T05:53:46.512982+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:50:11.678522+0000","last_clean_scrub_stamp":"2024-05-08T05:50:11.678522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:44:46.049256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"78'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510399+0000","last_change":"2024-05-08T05:49:56.605790+0000","last_active":"2024-05-08T05:53:46.510399+0000","last_peered":"2024-05-08T05:53:46.510399+0000","last_clean":"2024-05-08T05:53:46.510399+0000","last_became_active":"2024-05-08T05:49:56.605471+0000","last_became_peered":"2024-05-08T05:49:56.605471+0000","last_unstale":"2024-05-08T05:53:46.510399+0000","last_undegraded":"2024-05-08T05:53:46.510399+0000","last_fullsized":"2024-05-08T05:53:46.510399+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:55.586493+0000","last_clean_scrub_stamp":"2024-05-08T05:49:55.586493+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:20.266718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.512099+0000","last_change":"2024-05-08T05:49:59.638981+0000","last_active":"2024-05-08T05:53:46.512099+0000","last_peered":"2024-05-08T05:53:46.512099+0000","last_clean":"2024-05-08T05:53:46.512099+0000","last_became_active":"2024-05-08T05:49:59.638633+0000","last_became_peered":"2024-05-08T05:49:59.638633+0000","last_unstale":"2024-05-08T05:53:46.512099+0000","last_undegraded":"2024-05-08T05:53:46.512099+0000","last_fullsized":"2024-05-08T05:53:46.512099+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:40.846080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"89'9","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615200+0000","last_change":"2024-05-08T05:49:59.645430+0000","last_active":"2024-05-08T05:53:46.615200+0000","last_peered":"2024-05-08T05:53:46.615200+0000","last_clean":"2024-05-08T05:53:46.615200+0000","last_became_active":"2024-05-08T05:49:59.643933+0000","last_became_peered":"2024-05-08T05:49:59.643933+0000","last_unstale":"2024-05-08T05:53:46.615200+0000","last_undegraded":"2024-05-08T05:53:46.615200+0000","last_fullsized":"2024-05-08T05:53:46.615200+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:26:05.707266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"89'12","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614303+0000","last_change":"2024-05-08T05:49:59.640356+0000","last_active":"2024-05-08T05:53:46.614303+0000","last_peered":"2024-05-08T05:53:46.614303+0000","last_clean":"2024-05-08T05:53:46.614303+0000","last_became_active":"2024-05-08T05:49:59.639924+0000","last_became_peered":"2024-05-08T05:49:59.639924+0000","last_unstale":"2024-05-08T05:53:46.614303+0000","last_undegraded":"2024-05-08T05:53:46.614303+0000","last_fullsized":"2024-05-08T05:53:46.614303+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:43.626755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.613822+0000","last_change":"2024-05-08T05:49:59.638797+0000","last_active":"2024-05-08T05:53:46.613822+0000","last_peered":"2024-05-08T05:53:46.613822+0000","last_clean":"2024-05-08T05:53:46.613822+0000","last_became_active":"2024-05-08T05:49:59.638428+0000","last_became_peered":"2024-05-08T05:49:59.638428+0000","last_unstale":"2024-05-08T05:53:46.613822+0000","last_undegraded":"2024-05-08T05:53:46.613822+0000","last_fullsized":"2024-05-08T05:53:46.613822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:28:01.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"89'7","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614418+0000","last_change":"2024-05-08T05:49:59.649071+0000","last_active":"2024-05-08T05:53:46.614418+0000","last_peered":"2024-05-08T05:53:46.614418+0000","last_clean":"2024-05-08T05:53:46.614418+0000","last_became_active":"2024-05-08T05:49:59.647827+0000","last_became_peered":"2024-05-08T05:49:59.647827+0000","last_unstale":"2024-05-08T05:53:46.614418+0000","last_undegraded":"2024-05-08T05:53:46.614418+0000","last_fullsized":"2024-05-08T05:53:46.614418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:56:34.002071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"89'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.510198+0000","last_change":"2024-05-08T05:49:59.650716+0000","last_active":"2024-05-08T05:53:46.510198+0000","last_peered":"2024-05-08T05:53:46.510198+0000","last_clean":"2024-05-08T05:53:46.510198+0000","last_became_active":"2024-05-08T05:49:59.648619+0000","last_became_peered":"2024-05-08T05:49:59.648619+0000","last_unstale":"2024-05-08T05:53:46.510198+0000","last_undegraded":"2024-05-08T05:53:46.510198+0000","last_fullsized":"2024-05-08T05:53:46.510198+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:23:18.087175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.614774+0000","last_change":"2024-05-08T05:49:59.638536+0000","last_active":"2024-05-08T05:53:46.614774+0000","last_peered":"2024-05-08T05:53:46.614774+0000","last_clean":"2024-05-08T05:53:46.614774+0000","last_became_active":"2024-05-08T05:49:59.638197+0000","last_became_peered":"2024-05-08T05:49:59.638197+0000","last_unstale":"2024-05-08T05:53:46.614774+0000","last_undegraded":"2024-05-08T05:53:46.614774+0000","last_fullsized":"2024-05-08T05:53:46.614774+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:37.404863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"89'15","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615168+0000","last_change":"2024-05-08T05:49:59.637589+0000","last_active":"2024-05-08T05:53:46.615168+0000","last_peered":"2024-05-08T05:53:46.615168+0000","last_clean":"2024-05-08T05:53:46.615168+0000","last_became_active":"2024-05-08T05:49:59.637270+0000","last_became_peered":"2024-05-08T05:49:59.637270+0000","last_unstale":"2024-05-08T05:53:46.615168+0000","last_undegraded":"2024-05-08T05:53:46.615168+0000","last_fullsized":"2024-05-08T05:53:46.615168+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:50:10.606343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"89'2","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616073+0000","last_change":"2024-05-08T05:49:59.639030+0000","last_active":"2024-05-08T05:53:46.616073+0000","last_peered":"2024-05-08T05:53:46.616073+0000","last_clean":"2024-05-08T05:53:46.616073+0000","last_became_active":"2024-05-08T05:49:59.638696+0000","last_became_peered":"2024-05-08T05:49:59.638696+0000","last_unstale":"2024-05-08T05:53:46.616073+0000","last_undegraded":"2024-05-08T05:53:46.616073+0000","last_fullsized":"2024-05-08T05:53:46.616073+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:00:25.763328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"89'6","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.513136+0000","last_change":"2024-05-08T05:49:59.641857+0000","last_active":"2024-05-08T05:53:46.513136+0000","last_peered":"2024-05-08T05:53:46.513136+0000","last_clean":"2024-05-08T05:53:46.513136+0000","last_became_active":"2024-05-08T05:49:59.641532+0000","last_became_peered":"2024-05-08T05:49:59.641532+0000","last_unstale":"2024-05-08T05:53:46.513136+0000","last_undegraded":"2024-05-08T05:53:46.513136+0000","last_fullsized":"2024-05-08T05:53:46.513136+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:03:24.298651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"89'8","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.616577+0000","last_change":"2024-05-08T05:50:34.307921+0000","last_active":"2024-05-08T05:53:46.616577+0000","last_peered":"2024-05-08T05:53:46.616577+0000","last_clean":"2024-05-08T05:53:46.616577+0000","last_became_active":"2024-05-08T05:50:34.307609+0000","last_became_peered":"2024-05-08T05:50:34.307609+0000","last_unstale":"2024-05-08T05:53:46.616577+0000","last_undegraded":"2024-05-08T05:53:46.616577+0000","last_fullsized":"2024-05-08T05:53:46.616577+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:45:44.081632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"89'10","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615118+0000","last_change":"2024-05-08T05:49:59.644396+0000","last_active":"2024-05-08T05:53:46.615118+0000","last_peered":"2024-05-08T05:53:46.615118+0000","last_clean":"2024-05-08T05:53:46.615118+0000","last_became_active":"2024-05-08T05:49:59.644061+0000","last_became_peered":"2024-05-08T05:49:59.644061+0000","last_unstale":"2024-05-08T05:53:46.615118+0000","last_undegraded":"2024-05-08T05:53:46.615118+0000","last_fullsized":"2024-05-08T05:53:46.615118+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:55.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"89'11","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.509824+0000","last_change":"2024-05-08T05:49:59.646107+0000","last_active":"2024-05-08T05:53:46.509824+0000","last_peered":"2024-05-08T05:53:46.509824+0000","last_clean":"2024-05-08T05:53:46.509824+0000","last_became_active":"2024-05-08T05:49:59.645773+0000","last_became_peered":"2024-05-08T05:49:59.645773+0000","last_unstale":"2024-05-08T05:53:46.509824+0000","last_undegraded":"2024-05-08T05:53:46.509824+0000","last_fullsized":"2024-05-08T05:53:46.509824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:43.361033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"89'8","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:53:46.615234+0000","last_change":"2024-05-08T05:49:59.643233+0000","last_active":"2024-05-08T05:53:46.615234+0000","last_peered":"2024-05-08T05:53:46.615234+0000","last_clean":"2024-05-08T05:53:46.615234+0000","last_became_active":"2024-05-08T05:49:59.642922+0000","last_became_peered":"2024-05-08T05:49:59.642922+0000","last_unstale":"2024-05-08T05:53:46.615234+0000","last_undegraded":"2024-05-08T05:53:46.615234+0000","last_fullsized":"2024-05-08T05:53:46.615234+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:49:58.609044+0000","last_clean_scrub_stamp":"2024-05-08T05:49:58.609044+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:50.139116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106929,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":794085,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":703,"num_read_kb":561,"num_write":338,"num_write_kb":2331,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":2382255,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":237,"ondisk_log_size":237,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":70,"seq":300647710781,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":524274,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877907011,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28168,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716984,"statfs":{"total":95995035648,"available":95966191616,"internally_reserved":0,"allocated":1171456,"data_stored":1510811,"data_compressed":86227,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":922168,"data_compressed":54671,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":53,"seq":227633266766,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":918094,"data_compressed":54671,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462994,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":1119320,"data_compressed":71025,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626520,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":520178,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":592017,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738487,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":520268,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":592051,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182504,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1241730,"data_compressed":69873,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986275,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":524296,"data_compressed":35382,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822749,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1443056,"data_compressed":86227,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35643,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":71873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":721552,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":328414,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":393820,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":393798,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":400265,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:53:58.375 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:53:58.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:58 smithi003 ceph-mon[30137]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:58.666 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:53:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:58 smithi089 ceph-mon[34495]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:58 smithi104 ceph-mon[34559]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:53:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:53:59 smithi003 ceph-mon[30137]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 83 KiB/s wr, 43 op/s 2024-05-08T05:54:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:53:59 smithi089 ceph-mon[34495]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 83 KiB/s wr, 43 op/s 2024-05-08T05:54:00.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:53:59 smithi104 ceph-mon[34559]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 83 KiB/s wr, 43 op/s 2024-05-08T05:54:00.345 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:54:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:54:00.020+0000 7fb875af8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:54:00.345 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:54:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:54:00.020+0000 7fb875af8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:54:00.345 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:54:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:54:00.020+0000 7fb875af8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:54:00.346 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:54:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:54:00.020+0000 7fb875af8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:54:00.346 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 05:54:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T05:54:00.020+0000 7fb875af8700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 -- 172.21.15.3:0/2857297578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398103750 msgr2=0x7fe398103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2857297578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398103750 0x7fe398103b30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3940099c0 tx=0x7fe394009cd0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 -- 172.21.15.3:0/2857297578 shutdown_connections 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2857297578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe398108850 0x7fe3981103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2857297578 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398103750 0x7fe398103b30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2857297578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe398104100 0x7fe398108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 -- 172.21.15.3:0/2857297578 >> 172.21.15.3:0/2857297578 conn(0x7fe39807a600 msgr2=0x7fe39807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 -- 172.21.15.3:0/2857297578 shutdown_connections 2024-05-08T05:54:00.616 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.614+0000 7fe3a015c700 1 -- 172.21.15.3:0/2857297578 wait complete. 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 Processor -- start 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 -- start start 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe398103750 0x7fe3980757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe398104100 0x7fe398075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 0x7fe398076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe398113690 con 0x7fe398108850 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe398113510 con 0x7fe398104100 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe3a015c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe398113810 con 0x7fe398103750 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 0x7fe398076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 0x7fe398076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45000/0 (socket says 172.21.15.3:45000) 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 -- 172.21.15.3:0/2167577221 learned_addr learned my addr 172.21.15.3:0/2167577221 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:00.617 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39def8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe398103750 0x7fe3980757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39d6f7700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe398104100 0x7fe398075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 -- 172.21.15.3:0/2167577221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe398103750 msgr2=0x7fe3980757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe398103750 0x7fe3980757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 -- 172.21.15.3:0/2167577221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe398104100 msgr2=0x7fe398075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe398104100 0x7fe398075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.615+0000 7fe39e6f9700 1 -- 172.21.15.3:0/2167577221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3981a8050 con 0x7fe398108850 2024-05-08T05:54:00.618 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.616+0000 7fe39e6f9700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 0x7fe398076220 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe390007ee0 tx=0x7fe39000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.616+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390004bb0 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.616+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981a8340 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.616+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe390004d50 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.616+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390005920 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.616+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a8880 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.617+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39810ba90 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.618+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fe39001a020 con 0x7fe398108850 2024-05-08T05:54:00.620 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.619+0000 7fe38effd700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe3840b2b10 0x7fe3840b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:00.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.619+0000 7fe39def8700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe3840b2b10 0x7fe3840b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:00.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.619+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7fe3900e21e0 con 0x7fe398108850 2024-05-08T05:54:00.621 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.620+0000 7fe39def8700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe3840b2b10 0x7fe3840b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe394009990 tx=0x7fe394005c40 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:00.624 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.624+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe390010070 con 0x7fe398108850 2024-05-08T05:54:00.885 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.885+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe39810b470 con 0x7fe398108850 2024-05-08T05:54:00.886 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.886+0000 7fe38effd700 1 -- 172.21.15.3:0/2167577221 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7fe3900cb070 con 0x7fe398108850 2024-05-08T05:54:00.887 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:54:00.887 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.888+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe3840b2b10 msgr2=0x7fe3840b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.888+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe3840b2b10 0x7fe3840b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe394009990 tx=0x7fe394005c40 comp rx=0 tx=0).stop 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.888+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 msgr2=0x7fe398076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.888+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 0x7fe398076220 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe390007ee0 tx=0x7fe39000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 shutdown_connections 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe398108850 0x7fe398076220 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe3840b2b10 0x7fe3840b4fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.889 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe398103750 0x7fe3980757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 --2- 172.21.15.3:0/2167577221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe398104100 0x7fe398075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 >> 172.21.15.3:0/2167577221 conn(0x7fe39807a600 msgr2=0x7fe398101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 shutdown_connections 2024-05-08T05:54:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:00.889+0000 7fe3a015c700 1 -- 172.21.15.3:0/2167577221 wait complete. 2024-05-08T05:54:00.890 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:54:01.698 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:54:01.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:01 smithi003 ceph-mon[30137]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-08T05:54:01.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:01 smithi003 ceph-mon[30137]: mgrmap e25: x(active, since 14s), standbys: y, z 2024-05-08T05:54:01.965 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:01 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2167577221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:01.966 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3887485906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8410cff0 msgr2=0x7f0c8410f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3887485906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8410cff0 0x7f0c8410f420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009ca0 tx=0x7f0c7c02a950 comp rx=0 tx=0).stop 2024-05-08T05:54:01.974 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3887485906 shutdown_connections 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3887485906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c8410cff0 0x7f0c8410f420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3887485906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c84104540 0x7f0c8410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3887485906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 0x7f0c84104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3887485906 >> 172.21.15.3:0/3887485906 conn(0x7f0c840789d0 msgr2=0x7f0c84078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3887485906 shutdown_connections 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3887485906 wait complete. 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.974+0000 7f0c8aed1640 1 Processor -- start 2024-05-08T05:54:01.975 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 -- start start 2024-05-08T05:54:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 0x7f0c84199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 0x7f0c841996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.977 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8410cff0 0x7f0c8419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84112c30 con 0x7f0c84104540 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84112ab0 con 0x7f0c8410cff0 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8aed1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84112db0 con 0x7f0c84103c20 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 0x7f0c841996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 0x7f0c841996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45018/0 (socket says 172.21.15.3:45018) 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 -- 172.21.15.3:0/3193175207 learned_addr learned my addr 172.21.15.3:0/3193175207 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 0x7f0c84199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8946e640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8410cff0 0x7f0c8419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 -- 172.21.15.3:0/3193175207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 msgr2=0x7f0c84199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 0x7f0c84199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 -- 172.21.15.3:0/3193175207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8410cff0 msgr2=0x7f0c8419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8410cff0 0x7f0c8419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 -- 172.21.15.3:0/3193175207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8419e160 con 0x7f0c84104540 2024-05-08T05:54:01.978 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c8946e640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8410cff0 0x7f0c8419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 0x7f0c84199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.975+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 0x7f0c841996b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400d950 tx=0x7f0c7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.976+0000 7f0c79ffb640 1 -- 172.21.15.3:0/3193175207 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74017070 con 0x7f0c84104540 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.976+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8419e450 con 0x7f0c84104540 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.976+0000 7f0c79ffb640 1 -- 172.21.15.3:0/3193175207 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c74009c30 con 0x7f0c84104540 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.976+0000 7f0c79ffb640 1 -- 172.21.15.3:0/3193175207 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400f620 con 0x7f0c84104540 2024-05-08T05:54:01.979 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.976+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841a5ce0 con 0x7f0c84104540 2024-05-08T05:54:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.978+0000 7f0c79ffb640 1 -- 172.21.15.3:0/3193175207 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f0c74010460 con 0x7f0c84104540 2024-05-08T05:54:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 Processor -- start 2024-05-08T05:54:01.984 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 -- start start 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c79ffb640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0c6c0b2a70 0x7f0c6c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a3842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a38ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55580a2e95d0 con 0x55580a38f0f0 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55580a2a2890 con 0x55580a384830 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55580a2e5590 con 0x55580a383f10 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8946e640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0c6c0b2a70 0x7f0c6c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c79ffb640 1 -- 172.21.15.3:0/3193175207 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f0c740e2890 con 0x7f0c84104540 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8946e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8946e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3914e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45032/0 (socket says 172.21.15.3:45032) 2024-05-08T05:54:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c8946e640 1 -- 172.21.15.3:0/3091128346 learned_addr learned my addr 172.21.15.3:0/3091128346 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a3842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.979+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a38ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3091128346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 msgr2=0x55580a38ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a38ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3091128346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 msgr2=0x55580a3914e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3914e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3091128346 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555809ffb210 con 0x55580a383f10 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c8946e640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0c6c0b2a70 0x7f0c6c0b4f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c005fc0 tx=0x7f0c7c005f30 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a3842f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000cbd0 tx=0x7f0c8000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19703ed3ca990107 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c6affd640 1 -- 172.21.15.3:0/3091128346 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80005220 con 0x55580a383f10 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c6affd640 1 -- 172.21.15.3:0/3091128346 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c80004c50 con 0x55580a383f10 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.980+0000 7f0c6affd640 1 -- 172.21.15.3:0/3091128346 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80010690 con 0x55580a383f10 2024-05-08T05:54:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3091128346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 msgr2=0x55580a3842f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a3842f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000cbd0 tx=0x7f0c8000e720 comp rx=0 tx=0).stop 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3091128346 shutdown_connections 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3914e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a38ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3091128346 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a3842f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3091128346 >> 172.21.15.3:0/3091128346 conn(0x55580a380420 msgr2=0x55580a380800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3091128346 shutdown_connections 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3091128346 wait complete. 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.981+0000 7f0c8dbee740 1 -- start start 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 Processor -- start 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a383f10 0x55580a38bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a384830 0x55580a38c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a38ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55580a391d20 con 0x55580a38f0f0 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55580a2a6370 con 0x55580a383f10 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55580a391ea0 con 0x55580a384830 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8dbee740 10 client.0 _ll_register_callbacks cb 0x555809ffaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a38ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a38ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45042/0 (socket says 172.21.15.3:45042) 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 -- 172.21.15.3:0/1580215101 learned_addr learned my addr 172.21.15.3:0/1580215101 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a384830 0x55580a38c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a383f10 0x55580a38bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 -- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a384830 msgr2=0x55580a38c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a384830 0x55580a38c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 -- 172.21.15.3:0/1580215101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a383f10 msgr2=0x55580a38bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.982+0000 7f0c8946e640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a383f10 0x55580a38bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c8946e640 1 -- 172.21.15.3:0/1580215101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55580a0c7620 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c8946e640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a38ca10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006010 tx=0x7f0c7c047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25554bdcf07aae7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c63fff640 1 -- 172.21.15.3:0/1580215101 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c043040 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c63fff640 1 -- 172.21.15.3:0/1580215101 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f0c7c004480 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c80004c50 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c63fff640 1 -- 172.21.15.3:0/1580215101 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0c7c034040 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c8dbee740 10 client.15264 fetch_fsmap learned FSMap version 21 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55580a39dba0 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.983+0000 7f0c63fff640 1 -- 172.21.15.3:0/1580215101 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c80004c50 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c63fff640 1 -- 172.21.15.3:0/1580215101 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0c7c030cc0 con 0x55580a38f0f0 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c8dbee740 10 client.15264 fetch_fsmap finished waiting for FSMap version 21 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c8dbee740 10 client.15264 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c8dbee740 20 client.15264 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a3a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c8dbee740 4 client.15264 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55580a2f71e0 con 0x55580a39f0d0 2024-05-08T05:54:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a3a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a3a14e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77c380c7a9a58b2 server_cookie=af7453c23e7f3d3d in_seq=0 out_seq=0 2024-05-08T05:54:01.990 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.984+0000 7f0c63fff640 10 client.15264 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c63fff640 1 -- 172.21.15.3:0/1580215101 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55580a2f71e0 con 0x55580a39f0d0 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c63fff640 10 client.15264 handle_command_reply: tid=0 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 client.15264 shutdown 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 msgr2=0x55580a38ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a38ca10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c006010 tx=0x7f0c7c047000 comp rx=0 tx=0).stop 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 shutdown_connections 2024-05-08T05:54:01.991 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a3a14e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c63fff640 0 client.15264 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a38ca10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a384830 0x55580a38c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.985+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/1580215101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a383f10 0x55580a38bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.986+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 >> 172.21.15.3:0/1580215101 conn(0x55580a380420 msgr2=0x55580a386150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.986+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 shutdown_connections 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.986+0000 7f0c8dbee740 1 -- 172.21.15.3:0/1580215101 wait complete. 2024-05-08T05:54:01.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:01.986+0000 7f0c8dbee740 20 client.15264 trim_cache size 0 max 16384 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 Processor -- start 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 -- start start 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 0x55580a38b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a38f0f0 0x55580a3a7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841adc40 con 0x55580a383f10 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b04e0 con 0x55580a384830 2024-05-08T05:54:02.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841ae2d0 con 0x55580a38f0f0 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 0x55580a38b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 0x55580a38b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:45046/0 (socket says 172.21.15.3:45046) 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 -- 172.21.15.3:0/2088446650 learned_addr learned my addr 172.21.15.3:0/2088446650 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c8946e640 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a38f0f0 0x55580a3a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 -- 172.21.15.3:0/2088446650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a38f0f0 msgr2=0x55580a3a7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a38f0f0 0x55580a3a7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 -- 172.21.15.3:0/2088446650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 msgr2=0x55580a3a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.005+0000 7f0c88c6d640 1 -- 172.21.15.3:0/2088446650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55580a384660 con 0x55580a383f10 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.006+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 0x55580a38b0f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0c80005620 tx=0x7f0c800056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be6b3cd1eb386c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.006+0000 7f0c6a7fc640 1 -- 172.21.15.3:0/2088446650 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8002a320 con 0x55580a383f10 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.006+0000 7f0c6a7fc640 1 -- 172.21.15.3:0/2088446650 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8002a970 con 0x55580a383f10 2024-05-08T05:54:02.008 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.006+0000 7f0c6a7fc640 1 -- 172.21.15.3:0/2088446650 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80027b30 con 0x55580a383f10 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- 172.21.15.3:0/2088446650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 msgr2=0x55580a38b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 0x55580a38b0f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0c80005620 tx=0x7f0c800056f0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- 172.21.15.3:0/2088446650 shutdown_connections 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a38f0f0 0x55580a3a7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/2088446650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a383f10 0x55580a38b0f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- 172.21.15.3:0/2088446650 >> 172.21.15.3:0/2088446650 conn(0x55580a380420 msgr2=0x55580a37b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- 172.21.15.3:0/2088446650 shutdown_connections 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- 172.21.15.3:0/2088446650 wait complete. 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- start start 2024-05-08T05:54:02.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 Processor -- start 2024-05-08T05:54:02.012 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a389c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841ae9a0 con 0x55580a38f0f0 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841afe10 con 0x55580a384830 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.007+0000 7f0c8dbee740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841af070 con 0x55580a383f10 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c8dbee740 10 client.0 _ll_register_callbacks cb 0x7f0c7c009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a389c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a389c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:40880/0 (socket says 172.21.15.3:40880) 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3325870740 learned_addr learned my addr 172.21.15.3:0/3325870740 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c8946e640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3325870740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 msgr2=0x55580a3a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:02.013 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3325870740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 msgr2=0x55580a3a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 -- 172.21.15.3:0/3325870740 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55580a40c9d0 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c8946e640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c88c6d640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a389c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000cbd0 tx=0x7f0c8000c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e579052d42bfad5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c63fff640 1 -- 172.21.15.3:0/3325870740 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800278b0 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c63fff640 1 -- 172.21.15.3:0/3325870740 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f0c80027a50 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.008+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c8002a970 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c63fff640 1 -- 172.21.15.3:0/3325870740 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0c80027c50 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 10 client.25024 fetch_fsmap learned FSMap version 21 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) -- 0x55580a40daf0 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c63fff640 1 -- 172.21.15.3:0/3325870740 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8002a970 con 0x55580a383f10 2024-05-08T05:54:02.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c63fff640 1 -- 172.21.15.3:0/3325870740 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 21) ==== 8330+0+0 (secure 0 0 0) 0x7f0c8001a580 con 0x55580a383f10 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 10 client.25024 fetch_fsmap finished waiting for FSMap version 21 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 10 client.25024 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 20 client.25024 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a410070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 4 client.25024 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55580a2f1050 con 0x55580a39f0d0 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.009+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a410070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.010+0000 7f0c7bfff640 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a410070 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee689f00739efe54 server_cookie=9010ee8a4ef9398 in_seq=0 out_seq=0 2024-05-08T05:54:02.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:02.010+0000 7f0c63fff640 10 client.25024 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:02.016 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:01 smithi089 ceph-mon[34495]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-08T05:54:02.016 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:01 smithi089 ceph-mon[34495]: mgrmap e25: x(active, since 14s), standbys: y, z 2024-05-08T05:54:02.016 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:01 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2167577221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:02.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:01 smithi104 ceph-mon[34559]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-05-08T05:54:02.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:01 smithi104 ceph-mon[34559]: mgrmap e25: x(active, since 14s), standbys: y, z 2024-05-08T05:54:02.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:01 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2167577221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c63fff640 1 -- 172.21.15.3:0/3325870740 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55580a2f1050 con 0x55580a39f0d0 2024-05-08T05:54:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c63fff640 10 client.25024 handle_command_reply: tid=0 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 client.25024 shutdown 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 msgr2=0x55580a389c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a389c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000cbd0 tx=0x7f0c8000c310 comp rx=0 tx=0).stop 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 shutdown_connections 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55580a39f0d0 0x55580a410070 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55580a38f0f0 0x55580a3a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55580a384830 0x55580a3a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 --2- 172.21.15.3:0/3325870740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55580a383f10 0x55580a389c70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c63fff640 0 client.25024 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 >> 172.21.15.3:0/3325870740 conn(0x55580a380420 msgr2=0x55580a38b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 shutdown_connections 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.498+0000 7f0c8dbee740 1 -- 172.21.15.3:0/3325870740 wait complete. 2024-05-08T05:54:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8dbee740 20 client.25024 trim_cache size 0 max 16384 2024-05-08T05:54:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:54:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:54:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:54:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0c6c0b2a70 msgr2=0x7f0c6c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0c6c0b2a70 0x7f0c6c0b4f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c005fc0 tx=0x7f0c7c005f30 comp rx=0 tx=0).stop 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 msgr2=0x7f0c841996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 0x7f0c841996b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400d950 tx=0x7f0c7400de10 comp rx=0 tx=0).stop 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 shutdown_connections 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.499+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0c6c0b2a70 0x7f0c6c0b4f30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.500+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0c8410cff0 0x7f0c8419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.500+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0c84104540 0x7f0c841996b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.500+0000 7f0c8aed1640 1 --2- 172.21.15.3:0/3193175207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c84103c20 0x7f0c84199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.500+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 >> 172.21.15.3:0/3193175207 conn(0x7f0c840789d0 msgr2=0x7f0c840fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:03.502 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.500+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 shutdown_connections 2024-05-08T05:54:03.503 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:03.500+0000 7f0c8aed1640 1 -- 172.21.15.3:0/3193175207 wait complete. 2024-05-08T05:54:03.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:03 smithi003 ceph-mon[30137]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 87 KiB/s wr, 53 op/s 2024-05-08T05:54:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 -- 172.21.15.3:0/278616768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 msgr2=0x7f932406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 --2- 172.21.15.3:0/278616768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932406b600 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9318009990 tx=0x7f9318009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 -- 172.21.15.3:0/278616768 shutdown_connections 2024-05-08T05:54:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 --2- 172.21.15.3:0/278616768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f932406bb40 0x7f9324110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.792 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 --2- 172.21.15.3:0/278616768 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324109510 0x7f93241098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 --2- 172.21.15.3:0/278616768 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932406b600 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 -- 172.21.15.3:0/278616768 >> 172.21.15.3:0/278616768 conn(0x7f9324079350 msgr2=0x7f9324079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 -- 172.21.15.3:0/278616768 shutdown_connections 2024-05-08T05:54:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.791+0000 7f9329fa0700 1 -- 172.21.15.3:0/278616768 wait complete. 2024-05-08T05:54:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.792+0000 7f9329fa0700 1 Processor -- start 2024-05-08T05:54:03.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.792+0000 7f9329fa0700 1 -- start start 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f932406bb40 0x7f932419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324109510 0x7f93241a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9324113640 con 0x7f932406b180 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f93241134c0 con 0x7f9324109510 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f93241137c0 con 0x7f932406bb40 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932419bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35414/0 (socket says 172.21.15.3:35414) 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 -- 172.21.15.3:0/424183012 learned_addr learned my addr 172.21.15.3:0/424183012 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9322ffd700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f932406bb40 0x7f932419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:03.795 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9323fff700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324109510 0x7f93241a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 -- 172.21.15.3:0/424183012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f932406bb40 msgr2=0x7f932419c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f932406bb40 0x7f932419c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 -- 172.21.15.3:0/424183012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324109510 msgr2=0x7f93241a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324109510 0x7f93241a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 -- 172.21.15.3:0/424183012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93241a0d60 con 0x7f932406b180 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f93237fe700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932419bd50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f931400d950 tx=0x7f931400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9314016070 con 0x7f932406b180 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f931400fd40 con 0x7f932406b180 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.793+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93241a1050 con 0x7f932406b180 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.794+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931400e3b0 con 0x7f932406b180 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.794+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93241a88d0 con 0x7f932406b180 2024-05-08T05:54:03.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.794+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932406a190 con 0x7f932406b180 2024-05-08T05:54:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.796+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f9314019070 con 0x7f932406b180 2024-05-08T05:54:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.797+0000 7f9320ff9700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f930c0b2b10 0x7f930c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.797+0000 7f9322ffd700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f930c0b2b10 0x7f930c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.797+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7f93140e26c0 con 0x7f932406b180 2024-05-08T05:54:03.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.797+0000 7f9322ffd700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f930c0b2b10 0x7f930c0b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f93180098e0 tx=0x7f9318005960 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:03.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:03.801+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93140a9aa0 con 0x7f932406b180 2024-05-08T05:54:03.853 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-mon[34495]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 87 KiB/s wr, 53 op/s 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: *** Caught signal (Segmentation fault) ** 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: in thread 7fe215483700 thread_name:md_log_replay 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe226845d20] 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2: /usr/bin/ceph-mds(+0x54b9a2) [0x55b5b59b89a2] 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b5b59bc153] 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 4: (MDLog::_replay_thread()+0x617) [0x55b5b5a16c97] 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5b5a20381] 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe22683b1ca] 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 7: clone() 2024-05-08T05:54:03.854 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:03.837+0000 7fe215483700 -1 *** Caught signal (Segmentation fault) ** 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: in thread 7fe215483700 thread_name:md_log_replay 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe226845d20] 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2: /usr/bin/ceph-mds(+0x54b9a2) [0x55b5b59b89a2] 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b5b59bc153] 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 4: (MDLog::_replay_thread()+0x617) [0x55b5b5a16c97] 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5b5a20381] 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe22683b1ca] 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 7: clone() 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 0> 2024-05-08T05:54:03.837+0000 7fe215483700 -1 *** Caught signal (Segmentation fault) ** 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: in thread 7fe215483700 thread_name:md_log_replay 2024-05-08T05:54:03.855 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe226845d20] 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2: /usr/bin/ceph-mds(+0x54b9a2) [0x55b5b59b89a2] 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b5b59bc153] 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 4: (MDLog::_replay_thread()+0x617) [0x55b5b5a16c97] 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5b5a20381] 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe22683b1ca] 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 7: clone() 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-08T05:54:03.856 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:03 smithi104 ceph-mon[34559]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 87 KiB/s wr, 53 op/s 2024-05-08T05:54:04.063 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.063+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93241a8aa0 con 0x7f932406b180 2024-05-08T05:54:04.064 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.064+0000 7f9320ff9700 1 -- 172.21.15.3:0/424183012 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7f93140a93a0 con 0x7f932406b180 2024-05-08T05:54:04.064 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:54:04.064 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:54:04.066 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f930c0b2b10 msgr2=0x7f930c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f930c0b2b10 0x7f930c0b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f93180098e0 tx=0x7f9318005960 comp rx=0 tx=0).stop 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 msgr2=0x7f932419bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932419bd50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f931400d950 tx=0x7f931400dc60 comp rx=0 tx=0).stop 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 shutdown_connections 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9324109510 0x7f93241a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f932406b180 0x7f932419bd50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f930c0b2b10 0x7f930c0b4fd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 --2- 172.21.15.3:0/424183012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f932406bb40 0x7f932419c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 >> 172.21.15.3:0/424183012 conn(0x7f9324079350 msgr2=0x7f932410e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 shutdown_connections 2024-05-08T05:54:04.067 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:04.066+0000 7f9329fa0700 1 -- 172.21.15.3:0/424183012 wait complete. 2024-05-08T05:54:04.068 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:54:04.152 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 0> 2024-05-08T05:54:03.837+0000 7fe215483700 -1 *** Caught signal (Segmentation fault) ** 2024-05-08T05:54:04.152 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: in thread 7fe215483700 thread_name:md_log_replay 2024-05-08T05:54:04.152 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe226845d20] 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2: /usr/bin/ceph-mds(+0x54b9a2) [0x55b5b59b89a2] 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55b5b59bc153] 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 4: (MDLog::_replay_thread()+0x617) [0x55b5b5a16c97] 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55b5b5a20381] 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe22683b1ca] 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 7: clone() 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[65134]: 2024-05-08T05:54:04.153 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:03 smithi089 podman[68545]: 2024-05-08 05:54:03.899000846 +0000 UTC m=+0.022157582 container died 6358f2ed239491acb6b6b9fe3be123703c3f44e9e01ea7f3b99c3c419c56c614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-08T05:54:04.621 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:04 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/424183012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:04.652 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:54:04.900 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:04 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/424183012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:04.900 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:04 smithi089 podman[68545]: 2024-05-08 05:54:04.6537824 +0000 UTC m=+0.776939134 container remove 6358f2ed239491acb6b6b9fe3be123703c3f44e9e01ea7f3b99c3c419c56c614 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:54:04.901 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:04 smithi089 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service: Main process exited, code=exited, status=139/n/a 2024-05-08T05:54:04.911 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:05.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:04 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/424183012' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:05.262 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:04 smithi089 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service: Failed with result 'exit-code'. 2024-05-08T05:54:05.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:05 smithi003 ceph-mon[30137]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-05-08T05:54:06.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:05 smithi089 ceph-mon[34495]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-05-08T05:54:06.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:05 smithi104 ceph-mon[34559]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 62 KiB/s wr, 32 op/s 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 -- 172.21.15.3:0/3922622171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd668075ff0 msgr2=0x7fd668114160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 --2- 172.21.15.3:0/3922622171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd668075ff0 0x7fd668114160 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd658009990 tx=0x7fd658009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 -- 172.21.15.3:0/3922622171 shutdown_connections 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 --2- 172.21.15.3:0/3922622171 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd6681146a0 0x7fd668116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 --2- 172.21.15.3:0/3922622171 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6680756d0 0x7fd668075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 --2- 172.21.15.3:0/3922622171 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd668075ff0 0x7fd668114160 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 -- 172.21.15.3:0/3922622171 >> 172.21.15.3:0/3922622171 conn(0x7fd66807a610 msgr2=0x7fd66807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:06.700 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 -- 172.21.15.3:0/3922622171 shutdown_connections 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 -- 172.21.15.3:0/3922622171 wait complete. 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 Processor -- start 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.698+0000 7fd670342700 1 -- start start 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd670342700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6680756d0 0x7fd6681a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd670342700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd668075ff0 0x7fd6681a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd670342700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 0x7fd6681a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd670342700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd668119d70 con 0x7fd6681146a0 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd670342700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd668119bf0 con 0x7fd668075ff0 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd670342700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd668119ef0 con 0x7fd6680756d0 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 0x7fd6681a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 0x7fd6681a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35436/0 (socket says 172.21.15.3:35436) 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 -- 172.21.15.3:0/741771851 learned_addr learned my addr 172.21.15.3:0/741771851 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66d8dd700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd668075ff0 0x7fd6681a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:06.701 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e0de700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6680756d0 0x7fd6681a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 -- 172.21.15.3:0/741771851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6680756d0 msgr2=0x7fd6681a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6680756d0 0x7fd6681a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 -- 172.21.15.3:0/741771851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd668075ff0 msgr2=0x7fd6681a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd668075ff0 0x7fd6681a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 -- 172.21.15.3:0/741771851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6681a52b0 con 0x7fd6681146a0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.699+0000 7fd66e8df700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 0x7fd6681a4b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b9e0 tx=0x7fd66000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.700+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000d8e0 con 0x7fd6681146a0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.700+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681a55a0 con 0x7fd6681146a0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.700+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd660015420 con 0x7fd6681146a0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.700+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660014630 con 0x7fd6681146a0 2024-05-08T05:54:06.702 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.700+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681ace20 con 0x7fd6681146a0 2024-05-08T05:54:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.701+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66806a190 con 0x7fd6681146a0 2024-05-08T05:54:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.702+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fd6600155d0 con 0x7fd6681146a0 2024-05-08T05:54:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.703+0000 7fd65f7fe700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd6540b2b90 0x7fd6540b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.703+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7fd6600e2980 con 0x7fd6681146a0 2024-05-08T05:54:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.703+0000 7fd66e0de700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd6540b2b90 0x7fd6540b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:06.703 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.703+0000 7fd66e0de700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd6540b2b90 0x7fd6540b5050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd664004630 tx=0x7fd66400b3c0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:06.706 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.706+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd6600a9da0 con 0x7fd6681146a0 2024-05-08T05:54:06.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.923+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6681ad050 con 0x7fd6681146a0 2024-05-08T05:54:06.925 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.925+0000 7fd65f7fe700 1 -- 172.21.15.3:0/741771851 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25037 (secure 0 0 0) 0x7fd6600a96a0 con 0x7fd6681146a0 2024-05-08T05:54:06.925 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:54:06.926 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":90,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T05:53:46.504319+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2793699832":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/864931570":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/3161157856":"2024-05-09T05:53:46.504286+0000","172.21.15.3:6800/2793699832":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/1128678241":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/3921401880":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:54:06.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd6540b2b90 msgr2=0x7fd6540b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:06.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd6540b2b90 0x7fd6540b5050 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd664004630 tx=0x7fd66400b3c0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 msgr2=0x7fd6681a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 0x7fd6681a4b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd66000b9e0 tx=0x7fd66000bcf0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 shutdown_connections 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd6681146a0 0x7fd6681a4b70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd6540b2b90 0x7fd6540b5050 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6680756d0 0x7fd6681a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 --2- 172.21.15.3:0/741771851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd668075ff0 0x7fd6681a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 >> 172.21.15.3:0/741771851 conn(0x7fd66807a610 msgr2=0x7fd668110140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 shutdown_connections 2024-05-08T05:54:06.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:06.927+0000 7fd670342700 1 -- 172.21.15.3:0/741771851 wait complete. 2024-05-08T05:54:07.739 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-08T05:54:07.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:07 smithi003 ceph-mon[30137]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T05:54:07.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:07 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/741771851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:08.001 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:08.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:07 smithi089 ceph-mon[34495]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T05:54:08.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:07 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/741771851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:08.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:07 smithi104 ceph-mon[34559]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-05-08T05:54:08.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:07 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/741771851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:54:09.773 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:09 smithi003 ceph-mon[30137]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-05-08T05:54:10.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:09 smithi089 ceph-mon[34495]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-05-08T05:54:10.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:09 smithi104 ceph-mon[34559]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-05-08T05:54:10.603 INFO:teuthology.orchestra.run.smithi003.stdout:{"run_dir":"/var/run/ceph"} 2024-05-08T05:54:10.603 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-08T05:54:10.604 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5998 2024-05-08T05:54:10.604 DEBUG:tasks.quiescer:Yielding 2024-05-08T05:54:10.604 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T05:54:10.620 INFO:tasks.workunit:Pulling workunits from ref bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:54:10.621 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T05:54:10.621 INFO:tasks.workunit:timeout=3h 2024-05-08T05:54:10.621 INFO:tasks.workunit:cleanup=True 2024-05-08T05:54:10.621 DEBUG:teuthology.orchestra.run.smithi003:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:10.622 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:54:10.642 INFO:teuthology.orchestra.run.smithi003.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:10.642 INFO:teuthology.orchestra.run.smithi003.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T05:54:10.642 INFO:teuthology.orchestra.run.smithi003.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-08T05:54:10.642 INFO:teuthology.orchestra.run.smithi003.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:54:10.642 INFO:teuthology.orchestra.run.smithi003.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-08T05:54:10.643 INFO:teuthology.orchestra.run.smithi003.stdout:Access: 2024-05-08 05:51:17.513439264 +0000 2024-05-08T05:54:10.643 INFO:teuthology.orchestra.run.smithi003.stdout:Modify: 2024-05-08 05:51:17.513439264 +0000 2024-05-08T05:54:10.643 INFO:teuthology.orchestra.run.smithi003.stdout:Change: 2024-05-08 05:53:43.178403065 +0000 2024-05-08T05:54:10.643 INFO:teuthology.orchestra.run.smithi003.stdout: Birth: 2024-05-08 05:51:17.513439264 +0000 2024-05-08T05:54:10.643 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T05:54:10.644 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T05:54:10.717 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:54:10.775 INFO:tasks.workunit.client.0.smithi003.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T05:54:10.892 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:11.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:11 smithi003 ceph-mon[30137]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T05:54:12.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:11 smithi089 ceph-mon[34495]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T05:54:12.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:11 smithi104 ceph-mon[34559]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T05:54:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 -- 172.21.15.3:0/1485167237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 msgr2=0x7face81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 --2- 172.21.15.3:0/1485167237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face81103d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7face000b0d0 tx=0x7face000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.754 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 -- 172.21.15.3:0/1485167237 shutdown_connections 2024-05-08T05:54:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 --2- 172.21.15.3:0/1485167237 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face81103d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 --2- 172.21.15.3:0/1485167237 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 0x7face8103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 --2- 172.21.15.3:0/1485167237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face8104100 0x7face8108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 -- 172.21.15.3:0/1485167237 >> 172.21.15.3:0/1485167237 conn(0x7face807a600 msgr2=0x7face807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 -- 172.21.15.3:0/1485167237 shutdown_connections 2024-05-08T05:54:12.755 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.754+0000 7facede5c700 1 -- 172.21.15.3:0/1485167237 wait complete. 2024-05-08T05:54:12.756 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 Processor -- start 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 -- start start 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 0x7face819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face8104100 0x7face819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face8199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7face8113640 con 0x7face8108850 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.755+0000 7facede5c700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7face81134c0 con 0x7face8103750 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7facede5c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7face81137c0 con 0x7face8104100 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face8199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 0x7face819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face7fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face8199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58976/0 (socket says 172.21.15.3:58976) 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face7fff700 1 -- 172.21.15.3:0/4209166379 learned_addr learned my addr 172.21.15.3:0/4209166379 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7facdeffd700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face8104100 0x7face819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 -- 172.21.15.3:0/4209166379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face8104100 msgr2=0x7face819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face8104100 0x7face819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.757 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 -- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 msgr2=0x7face8199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face8199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 -- 172.21.15.3:0/4209166379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face8199fe0 con 0x7face8103750 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face7fff700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face8199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.756+0000 7face77fe700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 0x7face819f6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facd800e9f0 tx=0x7facd800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.757+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800c9c0 con 0x7face8103750 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.757+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face819a2d0 con 0x7face8103750 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.757+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81a6590 con 0x7face8103750 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.757+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd800cb60 con 0x7face8103750 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.757+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8018420 con 0x7face8103750 2024-05-08T05:54:12.758 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.757+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face806a190 con 0x7face8103750 2024-05-08T05:54:12.759 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.759+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7facd8010430 con 0x7face8103750 2024-05-08T05:54:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.760+0000 7face57fa700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7facd40b2b10 0x7facd40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.760+0000 7facdeffd700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7facd40b2b10 0x7facd40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:12.760 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.760+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9408+0+0 (secure 0 0 0) 0x7facd80e2fd0 con 0x7face8103750 2024-05-08T05:54:12.761 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.761+0000 7facdeffd700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7facd40b2b10 0x7facd40b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7face8103010 tx=0x7facd0009260 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:12.764 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:12.764+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7facd80aa3f0 con 0x7face8103750 2024-05-08T05:54:13.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.031+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7face802e510 con 0x7face8103750 2024-05-08T05:54:13.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.032+0000 7face57fa700 1 -- 172.21.15.3:0/4209166379 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+11797 (secure 0 0 0) 0x7facd80a9cf0 con 0x7face8103750 2024-05-08T05:54:13.033 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:54:13.033 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:51:05.493748+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/472688413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":472688413},{"type":"v1","addr":"172.21.15.89:6839","nonce":472688413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:54:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7facd40b2b10 msgr2=0x7facd40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7facd40b2b10 0x7facd40b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7face8103010 tx=0x7facd0009260 comp rx=0 tx=0).stop 2024-05-08T05:54:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 msgr2=0x7face819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 0x7face819f6c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facd800e9f0 tx=0x7facd800ed00 comp rx=0 tx=0).stop 2024-05-08T05:54:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 shutdown_connections 2024-05-08T05:54:13.035 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7face8108850 0x7face8199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7face8103750 0x7face819f6c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7facd40b2b10 0x7facd40b4fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 --2- 172.21.15.3:0/4209166379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7face8104100 0x7face819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 >> 172.21.15.3:0/4209166379 conn(0x7face807a600 msgr2=0x7face8101720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 shutdown_connections 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:13.035+0000 7facede5c700 1 -- 172.21.15.3:0/4209166379 wait complete. 2024-05-08T05:54:13.036 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 21 2024-05-08T05:54:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:13 smithi003 ceph-mon[30137]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T05:54:13.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:13 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/4209166379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:13.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:50:01.809365+0000', 'modified': '2024-05-08T05:51:05.493748+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.89:6839/472688413', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.89:6838', 'nonce': 472688413}, {'type': 'v1', 'addr': '172.21.15.89:6839', 'nonce': 472688413}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.104:6839/3320987620', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 3320987620}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 3320987620}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24493, 'qdb_cluster': [24493]} 2024-05-08T05:54:13.846 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:54:13.846 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:54:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:13 smithi089 ceph-mon[34495]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T05:54:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:13 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/4209166379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:14.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:13 smithi104 ceph-mon[34559]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-05-08T05:54:14.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:13 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/4209166379' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:14.147 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:15.384 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service: Scheduled restart job, restart counter is at 1. 2024-05-08T05:54:15.384 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 systemd[1]: Stopped Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:54:15.384 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 systemd[1]: Starting Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T05:54:15.384 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 podman[68646]: 2024-05-08 05:54:15.154338662 +0000 UTC m=+0.018238596 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:54:15.384 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 podman[68646]: 2024-05-08 05:54:15.262658292 +0000 UTC m=+0.126558268 container create 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:54:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:15 smithi089 ceph-mon[34495]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 14 KiB/s wr, 6 op/s 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:15 smithi089 ceph-mon[34495]: daemon mds.b restarted 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 podman[68646]: 2024-05-08 05:54:15.498561521 +0000 UTC m=+0.362461454 container init 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 podman[68646]: 2024-05-08 05:54:15.501018557 +0000 UTC m=+0.364918491 container start 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux ) 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 ceph-mds[68666]: -- 172.21.15.89:0/607769987 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bf13ab3a40 con 0x55bf13a86800 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[68662]: starting mds.b at 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 bash[68646]: 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 2024-05-08T05:54:15.763 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 05:54:15 smithi089 systemd[1]: Started Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T05:54:15.854 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:15 smithi003 ceph-mon[30137]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 14 KiB/s wr, 6 op/s 2024-05-08T05:54:15.854 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:15 smithi003 ceph-mon[30137]: daemon mds.b restarted 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.961+0000 7f7ada49f700 1 -- 172.21.15.3:0/3073303226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 msgr2=0x7f7ad4102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.961+0000 7f7ada49f700 1 --2- 172.21.15.3:0/3073303226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4102a10 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40099c0 tx=0x7f7ac4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 -- 172.21.15.3:0/3073303226 shutdown_connections 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 --2- 172.21.15.3:0/3073303226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad41076c0 0x7f7ad410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 --2- 172.21.15.3:0/3073303226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4102a10 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 --2- 172.21.15.3:0/3073303226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad4102fe0 0x7f7ad4106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 -- 172.21.15.3:0/3073303226 >> 172.21.15.3:0/3073303226 conn(0x7f7ad4079350 msgr2=0x7f7ad4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:15.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 -- 172.21.15.3:0/3073303226 shutdown_connections 2024-05-08T05:54:15.963 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 -- 172.21.15.3:0/3073303226 wait complete. 2024-05-08T05:54:15.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 Processor -- start 2024-05-08T05:54:15.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.962+0000 7f7ada49f700 1 -- start start 2024-05-08T05:54:15.964 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ada49f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ada49f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad4102fe0 0x7f7ad41045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ada49f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad41076c0 0x7f7ad4104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ada49f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad4112550 con 0x7f7ad4102630 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ada49f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad41123d0 con 0x7f7ad4102fe0 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ada49f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad41126d0 con 0x7f7ad41076c0 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:58996/0 (socket says 172.21.15.3:58996) 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 -- 172.21.15.3:0/563378878 learned_addr learned my addr 172.21.15.3:0/563378878 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad4102fe0 0x7f7ad41045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad41076c0 0x7f7ad4104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 -- 172.21.15.3:0/563378878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad41076c0 msgr2=0x7f7ad4104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad41076c0 0x7f7ad4104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 -- 172.21.15.3:0/563378878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad4102fe0 msgr2=0x7f7ad41045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad4102fe0 0x7f7ad41045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 -- 172.21.15.3:0/563378878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad41a8110 con 0x7f7ad4102630 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.963+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4104080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40098e0 tx=0x7f7ac40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.964+0000 7f7ad17fa700 1 -- 172.21.15.3:0/563378878 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4027070 con 0x7f7ad4102630 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.964+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad41a83a0 con 0x7f7ad4102630 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.964+0000 7f7ad17fa700 1 -- 172.21.15.3:0/563378878 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac4003f20 con 0x7f7ad4102630 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.964+0000 7f7ad17fa700 1 -- 172.21.15.3:0/563378878 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4022ca0 con 0x7f7ad4102630 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.964+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad41a87f0 con 0x7f7ad4102630 2024-05-08T05:54:15.966 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.966+0000 7f7ad17fa700 1 -- 172.21.15.3:0/563378878 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f7ac402a070 con 0x7f7ad4102630 2024-05-08T05:54:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ad17fa700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7abc0b2b10 0x7f7abc0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.968 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7abc0b2b10 0x7f7abc0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ad17fa700 1 -- 172.21.15.3:0/563378878 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7ac40e3dd0 con 0x7f7ad4102630 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ae4d32b80 1 Processor -- start 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ae4d32b80 1 -- start start 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7abc0b2b10 0x7f7abc0b4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ad41022a0 tx=0x7f7ac800a380 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.967+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d653860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d6576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d6580b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d5a5810 con 0x55ed3d653da0 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d544770 con 0x55ed3d653480 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d5a17d0 con 0x55ed3d657c30 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d6576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d6576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59000/0 (socket says 172.21.15.3:59000) 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 -- 172.21.15.3:0/4010841488 learned_addr learned my addr 172.21.15.3:0/4010841488 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:15.969 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d653860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d6580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 -- 172.21.15.3:0/4010841488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 msgr2=0x55ed3d6580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d6580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 -- 172.21.15.3:0/4010841488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 msgr2=0x55ed3d653860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d653860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 -- 172.21.15.3:0/4010841488 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3d4f2880 con 0x55ed3d653da0 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d6576a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800fe40 tx=0x7f7ac8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17423e111ee8fd6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7aba7fc700 1 -- 172.21.15.3:0/4010841488 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8016070 con 0x55ed3d653da0 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7aba7fc700 1 -- 172.21.15.3:0/4010841488 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac8014070 con 0x55ed3d653da0 2024-05-08T05:54:15.970 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.968+0000 7f7aba7fc700 1 -- 172.21.15.3:0/4010841488 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8010d80 con 0x55ed3d653da0 2024-05-08T05:54:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4010841488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 msgr2=0x55ed3d6576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d6576a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7ac800fe40 tx=0x7f7ac8012000 comp rx=0 tx=0).stop 2024-05-08T05:54:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4010841488 shutdown_connections 2024-05-08T05:54:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d6580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d653860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.973 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4010841488 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d6576a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4010841488 >> 172.21.15.3:0/4010841488 conn(0x55ed3d57f0b0 msgr2=0x55ed3d64dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4010841488 shutdown_connections 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.969+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4010841488 wait complete. 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 -- start start 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 Processor -- start 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d65ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d65c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d65c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d65fe40 con 0x55ed3d653da0 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d50bbe0 con 0x55ed3d653480 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d65ffc0 con 0x55ed3d657c30 2024-05-08T05:54:15.974 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ae4d32b80 10 client.0 _ll_register_callbacks cb 0x55ed3d53d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d65c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d65c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59008/0 (socket says 172.21.15.3:59008) 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ad37fe700 1 -- 172.21.15.3:0/3727168905 learned_addr learned my addr 172.21.15.3:0/3727168905 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d65c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.970+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d65ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ad37fe700 1 -- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 msgr2=0x55ed3d65c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d65c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ad37fe700 1 -- 172.21.15.3:0/3727168905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 msgr2=0x55ed3d65ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d65ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ad37fe700 1 -- 172.21.15.3:0/3727168905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3d541640 con 0x55ed3d653da0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d65c310 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80045f0 tx=0x7f7ac8010700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358549b2ad61813 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ab37fe700 1 -- 172.21.15.3:0/3727168905 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac8034060 con 0x55ed3d653da0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ab37fe700 1 -- 172.21.15.3:0/3727168905 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7ac8010d70 con 0x55ed3d653da0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac8014070 con 0x55ed3d653da0 2024-05-08T05:54:15.975 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ab37fe700 1 -- 172.21.15.3:0/3727168905 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7ac803b050 con 0x55ed3d653da0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ae4d32b80 10 client.15303 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ed3d66a5a0 con 0x55ed3d653da0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.971+0000 7f7ab37fe700 1 -- 172.21.15.3:0/3727168905 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac8014070 con 0x55ed3d653da0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ab37fe700 1 -- 172.21.15.3:0/3727168905 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7ac802f910 con 0x55ed3d653da0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ae4d32b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ae4d32b80 10 client.15303 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ae4d32b80 20 client.15303 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d66dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ae4d32b80 4 client.15303 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed3d61f600 con 0x55ed3d66c7b0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.972+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d66dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.973+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d66dc20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a12c73f856362d7 server_cookie=979730892f8ccb49 in_seq=0 out_seq=0 2024-05-08T05:54:15.976 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.973+0000 7f7ab37fe700 10 client.15303 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:15.977 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ab37fe700 1 -- 172.21.15.3:0/3727168905 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed3d61f600 con 0x55ed3d66c7b0 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ab37fe700 10 client.15303 handle_command_reply: tid=0 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 client.15303 shutdown 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 msgr2=0x55ed3d65c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d65c310 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80045f0 tx=0x7f7ac8010700 comp rx=0 tx=0).stop 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 shutdown_connections 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d657c30 0x55ed3d65c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d66dc20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d653480 0x55ed3d65ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ab37fe700 0 client.15303 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/3727168905 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653da0 0x55ed3d65c310 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 >> 172.21.15.3:0/3727168905 conn(0x55ed3d57f0b0 msgr2=0x55ed3d6486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 shutdown_connections 2024-05-08T05:54:15.978 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/3727168905 wait complete. 2024-05-08T05:54:15.979 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.974+0000 7f7ae4d32b80 20 client.15303 trim_cache size 0 max 16384 2024-05-08T05:54:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.995+0000 7f7ae4d32b80 1 Processor -- start 2024-05-08T05:54:15.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.995+0000 7f7ae4d32b80 1 -- start start 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.995+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d65c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.995+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d648550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.995+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d6555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d66bf60 con 0x55ed3d653480 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d66c630 con 0x55ed3d657c30 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d66cd00 con 0x55ed3d653da0 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d65c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d648550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d65c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59014/0 (socket says 172.21.15.3:59014) 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 -- 172.21.15.3:0/629194706 learned_addr learned my addr 172.21.15.3:0/629194706 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d6555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 -- 172.21.15.3:0/629194706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 msgr2=0x55ed3d648550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:15.997 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d648550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 -- 172.21.15.3:0/629194706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 msgr2=0x55ed3d6555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d6555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 -- 172.21.15.3:0/629194706 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed3d64a3b0 con 0x55ed3d653480 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d6555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d65c880 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40aaad0 tx=0x7f7ac4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0ccfa875fa9ff3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ab9ffb700 1 -- 172.21.15.3:0/629194706 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40aa650 con 0x55ed3d653480 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ab9ffb700 1 -- 172.21.15.3:0/629194706 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac40ab240 con 0x55ed3d653480 2024-05-08T05:54:15.998 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.996+0000 7f7ab9ffb700 1 -- 172.21.15.3:0/629194706 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40ab000 con 0x55ed3d653480 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/629194706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 msgr2=0x55ed3d65c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d65c880 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40aaad0 tx=0x7f7ac4009310 comp rx=0 tx=0).stop 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/629194706 shutdown_connections 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d6555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d65c880 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/629194706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d648550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.002 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/629194706 >> 172.21.15.3:0/629194706 conn(0x55ed3d57f0b0 msgr2=0x55ed3d647e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/629194706 shutdown_connections 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/629194706 wait complete. 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.997+0000 7f7ae4d32b80 1 -- start start 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 Processor -- start 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d670c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d6711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d671700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d66daa0 con 0x55ed3d653480 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d66d3d0 con 0x55ed3d657c30 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ed3d66e170 con 0x55ed3d653da0 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ae4d32b80 10 client.0 _ll_register_callbacks cb 0x55ed3d53d8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d6711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d670c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d670c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59024/0 (socket says 172.21.15.3:59024) 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad37fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d6711c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:33738/0 (socket says 172.21.15.3:33738) 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 -- 172.21.15.3:0/4283126049 learned_addr learned my addr 172.21.15.3:0/4283126049 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d671700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 -- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 msgr2=0x55ed3d6711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d6711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 -- 172.21.15.3:0/4283126049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 msgr2=0x55ed3d671700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d671700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 -- 172.21.15.3:0/4283126049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed3d6754f0 con 0x55ed3d653480 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad8a3c700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d671700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ad3fff700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d670c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40aa840 tx=0x7f7ac40e0fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a9ded61223682 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.998+0000 7f7ab37fe700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4283126049 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40aeef0 con 0x55ed3d653480 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4283126049 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7ac40aada0 con 0x55ed3d653480 2024-05-08T05:54:16.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ac40ab240 con 0x55ed3d653480 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4283126049 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7ac40b3910 con 0x55ed3d653480 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 10 client.15309 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ed3d6d7fa0 con 0x55ed3d653480 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4283126049 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ac40ab240 con 0x55ed3d653480 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4283126049 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7ac40e12d0 con 0x55ed3d653480 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 10 client.15309 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 10 client.15309 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 20 client.15309 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d6db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 4 client.15309 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "status"}) -- 0x55ed3d5b31f0 con 0x55ed3d66c7b0 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:15.999+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d6db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ad37fe700 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d6db670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2e5a42e797226c server_cookie=db1ccec41425c38c in_seq=0 out_seq=0 2024-05-08T05:54:16.005 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ab37fe700 10 client.15309 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ab37fe700 1 -- 172.21.15.3:0/4283126049 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ed3d5b31f0 con 0x55ed3d66c7b0 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ab37fe700 10 client.15309 handle_command_reply: tid=0 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 client.15309 shutdown 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 msgr2=0x55ed3d670c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d670c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40aa840 tx=0x7f7ac40e0fe0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 shutdown_connections 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ed3d66c7b0 0x55ed3d6db670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed3d657c30 0x55ed3d671700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ed3d653480 0x55ed3d670c80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 --2- 172.21.15.3:0/4283126049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ed3d653da0 0x55ed3d6711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 >> 172.21.15.3:0/4283126049 conn(0x55ed3d57f0b0 msgr2=0x55ed3d6486d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ab37fe700 0 client.15309 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:16.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 shutdown_connections 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 1 -- 172.21.15.3:0/4283126049 wait complete. 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.000+0000 7f7ae4d32b80 20 client.15309 trim_cache size 0 max 16384 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "cluster_fsid": "708b9316-0cfd-11ef-bc97-c7b262605968", 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "whoami": 0, 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "id": 24493, 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "want_state": "up:active", 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "state": "up:active", 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "fs_name": "cephfs", 2024-05-08T05:54:16.007 INFO:teuthology.orchestra.run.smithi003.stdout: "rank_uptime": 253.36130540299999, 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "mdsmap_epoch": 23, 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch": 91, 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_epoch_barrier": 91, 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stdout: "uptime": 282.51382651599999 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7abc0b2b10 msgr2=0x7f7abc0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7abc0b2b10 0x7f7abc0b4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ad41022a0 tx=0x7f7ac800a380 comp rx=0 tx=0).stop 2024-05-08T05:54:16.008 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 msgr2=0x7f7ad4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4104080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40098e0 tx=0x7f7ac40057c0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 shutdown_connections 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7abc0b2b10 0x7f7abc0b4fd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ad41076c0 0x7f7ad4104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7ad4102630 0x7f7ad4104080 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 --2- 172.21.15.3:0/563378878 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7ad4102fe0 0x7f7ad41045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 >> 172.21.15.3:0/563378878 conn(0x7f7ad4079350 msgr2=0x7f7ad406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 shutdown_connections 2024-05-08T05:54:16.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:16.001+0000 7f7ada49f700 1 -- 172.21.15.3:0/563378878 wait complete. 2024-05-08T05:54:16.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:15 smithi104 ceph-mon[34559]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 14 KiB/s wr, 6 op/s 2024-05-08T05:54:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:15 smithi104 ceph-mon[34559]: daemon mds.b restarted 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 -- 172.21.15.3:0/1999597669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 msgr2=0x7ff0d010f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/1999597669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d010f440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009be0 tx=0x7ff0c802a750 comp rx=0 tx=0).stop 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 -- 172.21.15.3:0/1999597669 shutdown_connections 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/1999597669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d010f440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/1999597669 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0104560 0x7ff0d010ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/1999597669 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d0103c40 0x7ff0d0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 -- 172.21.15.3:0/1999597669 >> 172.21.15.3:0/1999597669 conn(0x7ff0d0078a00 msgr2=0x7ff0d0078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 -- 172.21.15.3:0/1999597669 shutdown_connections 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.597+0000 7ff0d56fc640 1 -- 172.21.15.3:0/1999597669 wait complete. 2024-05-08T05:54:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 Processor -- start 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 -- start start 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 0x7ff0d0101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d0104560 0x7ff0d0101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d0102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0112c50 con 0x7ff0d0103c40 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0112ad0 con 0x7ff0d0104560 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.598+0000 7ff0d56fc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0112dd0 con 0x7ff0d010d010 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 0x7ff0d0101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 0x7ff0d0101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59040/0 (socket says 172.21.15.3:59040) 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 -- 172.21.15.3:0/3459816437 learned_addr learned my addr 172.21.15.3:0/3459816437 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d0102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d0104560 0x7ff0d0101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 -- 172.21.15.3:0/3459816437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 msgr2=0x7ff0d0102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d0102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 -- 172.21.15.3:0/3459816437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d0104560 msgr2=0x7ff0d0101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d0104560 0x7ff0d0101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 -- 172.21.15.3:0/3459816437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01a7610 con 0x7ff0d0103c40 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 0x7ff0d0101890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000d950 tx=0x7ff0c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0b7fff640 1 -- 172.21.15.3:0/3459816437 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0017070 con 0x7ff0d0103c40 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a7900 con 0x7ff0d0103c40 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0b7fff640 1 -- 172.21.15.3:0/3459816437 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c0009c30 con 0x7ff0d0103c40 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0b7fff640 1 -- 172.21.15.3:0/3459816437 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000f620 con 0x7ff0d0103c40 2024-05-08T05:54:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.599+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a7e50 con 0x7ff0d0103c40 2024-05-08T05:54:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.602+0000 7ff0d8411740 1 Processor -- start 2024-05-08T05:54:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 -- start start 2024-05-08T05:54:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0b7fff640 1 -- 172.21.15.3:0/3459816437 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7ff0c0010460 con 0x7ff0d0103c40 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ad9f10 0x55dde1ada2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 0x55dde1ae4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ae50f0 0x55dde1ae74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dde1a3f5d0 con 0x55dde1ada830 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dde19f8890 con 0x55dde1ad9f10 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0d8411740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dde1a3b590 con 0x55dde1ae50f0 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 0x55dde1ae4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 0x55dde1ae4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59050/0 (socket says 172.21.15.3:59050) 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1594514226 learned_addr learned my addr 172.21.15.3:0/1594514226 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ae50f0 0x55dde1ae74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ad9f10 0x55dde1ada2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1594514226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ae50f0 msgr2=0x55dde1ae74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ae50f0 0x55dde1ae74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1594514226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ad9f10 msgr2=0x55dde1ada2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ad9f10 0x55dde1ada2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1594514226 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dde1751210 con 0x55dde1ada830 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.603+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ae50f0 0x55dde1ae74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 0x55dde1ae4bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00dfa70 tx=0x7ff0c00e8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3408007c22905083 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0b7fff640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff0b80b2a70 0x7ff0b80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff0b80b2a70 0x7ff0b80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0b57fa640 1 -- 172.21.15.3:0/1594514226 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00f1070 con 0x55dde1ada830 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0b7fff640 1 -- 172.21.15.3:0/3459816437 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff0c00e2870 con 0x7ff0d0103c40 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0b57fa640 1 -- 172.21.15.3:0/1594514226 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c00a90a0 con 0x55dde1ada830 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.604+0000 7ff0b57fa640 1 -- 172.21.15.3:0/1594514226 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c00a8d00 con 0x55dde1ada830 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- 172.21.15.3:0/1594514226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 msgr2=0x55dde1ae4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff0b80b2a70 0x7ff0b80b4f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006010 tx=0x7ff0c8004a40 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 0x55dde1ae4bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00dfa70 tx=0x7ff0c00e8f10 comp rx=0 tx=0).stop 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- 172.21.15.3:0/1594514226 shutdown_connections 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ae50f0 0x55dde1ae74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ada830 0x55dde1ae4bb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1594514226 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ad9f10 0x55dde1ada2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- 172.21.15.3:0/1594514226 >> 172.21.15.3:0/1594514226 conn(0x55dde1ad6420 msgr2=0x55dde1ad6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- 172.21.15.3:0/1594514226 shutdown_connections 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- 172.21.15.3:0/1594514226 wait complete. 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- start start 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 Processor -- start 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1ae2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1ae25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1ae2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dde1ae7d20 con 0x55dde1ae50f0 2024-05-08T05:54:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dde19fc370 con 0x55dde1ada830 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.605+0000 7ff0d8411740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dde1ae7ea0 con 0x55dde1ad9f10 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0d8411740 10 client.0 _ll_register_callbacks cb 0x55dde1750a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1ae2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0cf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1ae2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1ae2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:33768/0 (socket says 172.21.15.3:33768) 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1592749813 learned_addr learned my addr 172.21.15.3:0/1592749813 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1ae25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1592749813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 msgr2=0x55dde1ae25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1ae25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1592749813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 msgr2=0x55dde1ae2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1ae2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 -- 172.21.15.3:0/1592749813 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dde181d620 con 0x55dde1ad9f10 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1ae2070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00a9520 tx=0x7ff0c0010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30fd5d10cfdd4f77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/1592749813 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0020030 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c00a90a0 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/1592749813 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff0c00a97f0 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.606+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/1592749813 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff0c00f1070 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 10 client.25078 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dde1ae7d20 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/1592749813 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c00a90a0 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/1592749813 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7ff0c00b1f60 con 0x55dde1ad9f10 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 10 client.25078 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 10 client.25078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 20 client.25078 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1af4d90 0x55dde1af7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 4 client.25078 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dde1a4d1e0 con 0x55dde1af4d90 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.607+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1af4d90 0x55dde1af7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0b2ff5640 10 client.25078 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1af4d90 0x55dde1af7180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a461c4678de77 server_cookie=9df530f2a0eeecd0 in_seq=0 out_seq=0 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/1592749813 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dde1a4d1e0 con 0x55dde1af4d90 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0b2ff5640 10 client.25078 handle_command_reply: tid=0 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 client.25078 shutdown 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 msgr2=0x55dde1ae2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1ae2070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00a9520 tx=0x7ff0c0010cb0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 shutdown_connections 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1af4d90 0x55dde1af7180 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1ae2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1ae25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 --2- 172.21.15.3:0/1592749813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1ae2070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0b2ff5640 0 client.25078 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 >> 172.21.15.3:0/1592749813 conn(0x55dde1ad6420 msgr2=0x55dde1adc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.608+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 shutdown_connections 2024-05-08T05:54:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.609+0000 7ff0d8411740 1 -- 172.21.15.3:0/1592749813 wait complete. 2024-05-08T05:54:16.616 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.609+0000 7ff0d8411740 20 client.25078 trim_cache size 0 max 16384 2024-05-08T05:54:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 Processor -- start 2024-05-08T05:54:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 -- start start 2024-05-08T05:54:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 0x55dde1ad5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.628 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ada830 0x55dde1ae2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ae50f0 0x55dde1afc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01affb0 con 0x55dde1ad9f10 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b2850 con 0x55dde1ae50f0 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0d8411740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b0640 con 0x55dde1ada830 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 0x55dde1ad5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 0x55dde1ad5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59070/0 (socket says 172.21.15.3:59070) 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 -- 172.21.15.3:0/2301867757 learned_addr learned my addr 172.21.15.3:0/2301867757 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ada830 0x55dde1ae2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ae50f0 0x55dde1afc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 -- 172.21.15.3:0/2301867757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ada830 msgr2=0x55dde1ae2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ada830 0x55dde1ae2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 -- 172.21.15.3:0/2301867757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ae50f0 msgr2=0x55dde1afc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ae50f0 0x55dde1afc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.629 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 -- 172.21.15.3:0/2301867757 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dde1ae05c0 con 0x55dde1ad9f10 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.624+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 0x55dde1ad5580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00b2300 tx=0x7ff0c000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b203dd7a6f7678b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0b4ff9640 1 -- 172.21.15.3:0/2301867757 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0020070 con 0x55dde1ad9f10 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0b4ff9640 1 -- 172.21.15.3:0/2301867757 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c00dfa10 con 0x55dde1ad9f10 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0b4ff9640 1 -- 172.21.15.3:0/2301867757 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0016170 con 0x55dde1ad9f10 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 -- 172.21.15.3:0/2301867757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 msgr2=0x55dde1ad5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 0x55dde1ad5580 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00b2300 tx=0x7ff0c000df40 comp rx=0 tx=0).stop 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 -- 172.21.15.3:0/2301867757 shutdown_connections 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ae50f0 0x55dde1afc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ada830 0x55dde1ae2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 --2- 172.21.15.3:0/2301867757 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ad9f10 0x55dde1ad5580 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 -- 172.21.15.3:0/2301867757 >> 172.21.15.3:0/2301867757 conn(0x55dde1ad6420 msgr2=0x55dde1ae38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 -- 172.21.15.3:0/2301867757 shutdown_connections 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 -- 172.21.15.3:0/2301867757 wait complete. 2024-05-08T05:54:16.630 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.625+0000 7ff0d8411740 1 -- start start 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 Processor -- start 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1adef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1adf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1adf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b0d10 con 0x55dde1ae50f0 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b2180 con 0x55dde1ada830 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b13e0 con 0x55dde1ad9f10 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0d8411740 10 client.0 _ll_register_callbacks cb 0x7ff0c4009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1adf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1adf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:59074/0 (socket says 172.21.15.3:59074) 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 -- 172.21.15.3:0/4056824674 learned_addr learned my addr 172.21.15.3:0/4056824674 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1adef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1adf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 -- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 msgr2=0x55dde1adef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.631 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1adef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 -- 172.21.15.3:0/4056824674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 msgr2=0x55dde1adf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1adf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 -- 172.21.15.3:0/4056824674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dde1b5e4a0 con 0x55dde1ae50f0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1adef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0ce7fc640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1adf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0cf7fe640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1adf9d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8033000 tx=0x7ff0c802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7b3d1b97e7160a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.626+0000 7ff0b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/4056824674 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8036040 con 0x55dde1ae50f0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/4056824674 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff0c8031940 con 0x55dde1ae50f0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c00dfa10 con 0x55dde1ae50f0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/4056824674 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff0c805f050 con 0x55dde1ae50f0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0d8411740 10 client.15327 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dde1b5f700 con 0x55dde1ae50f0 2024-05-08T05:54:16.632 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/4056824674 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c00dfa10 con 0x55dde1ae50f0 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/4056824674 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7ff0c804d440 con 0x55dde1ae50f0 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0d8411740 10 client.15327 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.627+0000 7ff0d8411740 10 client.15327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0d8411740 20 client.15327 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0d8411740 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1b5fe80 0x55dde1b622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0d8411740 4 client.15327 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dde1a47050 con 0x55dde1b5fe80 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1b5fe80 0x55dde1b622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0b2ff5640 10 client.15327 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:16.633 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.628+0000 7ff0ceffd640 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1b5fe80 0x55dde1b622a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9424baa79e5af0 server_cookie=c94c2d7f2b5082ae in_seq=0 out_seq=0 2024-05-08T05:54:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.639+0000 7ff0b2ff5640 1 -- 172.21.15.3:0/4056824674 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dde1a47050 con 0x55dde1b5fe80 2024-05-08T05:54:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.639+0000 7ff0b2ff5640 10 client.15327 handle_command_reply: tid=0 2024-05-08T05:54:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 client.15327 shutdown 2024-05-08T05:54:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 msgr2=0x55dde1adf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.640 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1adf9d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8033000 tx=0x7ff0c802a750 comp rx=0 tx=0).stop 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 shutdown_connections 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dde1b5fe80 0x55dde1b622a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dde1ae50f0 0x55dde1adf9d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dde1ada830 0x55dde1adf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0b2ff5640 0 client.15327 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 --2- 172.21.15.3:0/4056824674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dde1ad9f10 0x55dde1adef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 >> 172.21.15.3:0/4056824674 conn(0x55dde1ad6420 msgr2=0x55dde1ad5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 shutdown_connections 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 1 -- 172.21.15.3:0/4056824674 wait complete. 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d8411740 20 client.15327 trim_cache size 0 max 16384 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:54:16.641 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:54:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:54:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:54:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff0b80b2a70 msgr2=0x7ff0b80b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.642 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff0b80b2a70 0x7ff0b80b4f30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8006010 tx=0x7ff0c8004a40 comp rx=0 tx=0).stop 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 msgr2=0x7ff0d0101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.640+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 0x7ff0d0101890 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000d950 tx=0x7ff0c000de10 comp rx=0 tx=0).stop 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 shutdown_connections 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff0b80b2a70 0x7ff0b80b4f30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0d010d010 0x7ff0d0102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff0d0104560 0x7ff0d0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 --2- 172.21.15.3:0/3459816437 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff0d0103c40 0x7ff0d0101890 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 >> 172.21.15.3:0/3459816437 conn(0x7ff0d0078a00 msgr2=0x7ff0d00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 shutdown_connections 2024-05-08T05:54:16.643 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:16.641+0000 7ff0d56fc640 1 -- 172.21.15.3:0/3459816437 wait complete. 2024-05-08T05:54:16.834 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-08T05:54:16.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:16 smithi003 ceph-mon[30137]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T05:54:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:16 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.89:6838/3215381485,v1:172.21.15.89:6839/3215381485] up:boot 2024-05-08T05:54:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:16 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:54:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:16 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:54:16.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:16 smithi003 ceph-mon[30137]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:54:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:16 smithi089 ceph-mon[34495]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T05:54:17.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:16 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.89:6838/3215381485,v1:172.21.15.89:6839/3215381485] up:boot 2024-05-08T05:54:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:16 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:54:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:16 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:54:17.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:16 smithi089 ceph-mon[34495]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:54:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:16 smithi104 ceph-mon[34559]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T05:54:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:16 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.89:6838/3215381485,v1:172.21.15.89:6839/3215381485] up:boot 2024-05-08T05:54:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:16 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-05-08T05:54:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:16 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:54:17.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:16 smithi104 ceph-mon[34559]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-05-08T05:54:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:17 smithi003 ceph-mon[30137]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-05-08T05:54:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:17 smithi089 ceph-mon[34495]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-05-08T05:54:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:17 smithi104 ceph-mon[34559]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 409 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-05-08T05:54:19.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:19 smithi003 ceph-mon[30137]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-05-08T05:54:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:19 smithi089 ceph-mon[34495]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-05-08T05:54:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:19 smithi104 ceph-mon[34559]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-05-08T05:54:21.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:21 smithi003 ceph-mon[30137]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-05-08T05:54:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:21 smithi089 ceph-mon[34495]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-05-08T05:54:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:21 smithi104 ceph-mon[34559]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.3 KiB/s wr, 4 op/s 2024-05-08T05:54:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:23 smithi003 ceph-mon[30137]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-08T05:54:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:23 smithi089 ceph-mon[34495]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-08T05:54:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:23 smithi104 ceph-mon[34559]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-05-08T05:54:25.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:25 smithi003 ceph-mon[30137]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-08T05:54:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:25 smithi089 ceph-mon[34495]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-08T05:54:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:25 smithi104 ceph-mon[34559]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-05-08T05:54:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:27 smithi003 ceph-mon[30137]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-05-08T05:54:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:27 smithi089 ceph-mon[34495]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-05-08T05:54:28.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:27 smithi104 ceph-mon[34559]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 -- 172.21.15.3:0/2304928154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528103370 msgr2=0x7fa52810f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 --2- 172.21.15.3:0/2304928154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528103370 0x7fa52810f880 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa520009cd0 tx=0x7fa52002a8e0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 -- 172.21.15.3:0/2304928154 shutdown_connections 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 --2- 172.21.15.3:0/2304928154 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa528103370 0x7fa52810f880 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 --2- 172.21.15.3:0/2304928154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa5281029b0 0x7fa528102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 --2- 172.21.15.3:0/2304928154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 0x7fa528108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 -- 172.21.15.3:0/2304928154 >> 172.21.15.3:0/2304928154 conn(0x7fa5280fe4a0 msgr2=0x7fa5281008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 -- 172.21.15.3:0/2304928154 shutdown_connections 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.743+0000 7fa530714640 1 -- 172.21.15.3:0/2304928154 wait complete. 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 Processor -- start 2024-05-08T05:54:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 -- start start 2024-05-08T05:54:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 0x7fa5281991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa528103370 0x7fa528199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 0x7fa52819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa528112c80 con 0x7fa5281029b0 2024-05-08T05:54:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa528112b00 con 0x7fa5281089c0 2024-05-08T05:54:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa530714640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa528112e00 con 0x7fa528103370 2024-05-08T05:54:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 0x7fa5281991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 0x7fa5281991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52172/0 (socket says 172.21.15.3:52172) 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2374028866 learned_addr learned my addr 172.21.15.3:0/2374028866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa528103370 0x7fa528199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 0x7fa52819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2374028866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa528103370 msgr2=0x7fa528199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa528103370 0x7fa528199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2374028866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 msgr2=0x7fa52819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.744+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 0x7fa52819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2374028866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52819e1b0 con 0x7fa5281029b0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 0x7fa52819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 0x7fa5281991c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9f0 tx=0x7fa52400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa51f7fe640 1 -- 172.21.15.3:0/2374028866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52400cd80 con 0x7fa5281029b0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa51f7fe640 1 -- 172.21.15.3:0/2374028866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5240108e0 con 0x7fa5281029b0 2024-05-08T05:54:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa51f7fe640 1 -- 172.21.15.3:0/2374028866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524018980 con 0x7fa5281029b0 2024-05-08T05:54:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52819e4a0 con 0x7fa5281029b0 2024-05-08T05:54:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.745+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a5d30 con 0x7fa5281029b0 2024-05-08T05:54:29.755 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.748+0000 7fa53342c740 1 Processor -- start 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 -- start start 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa51f7fe640 1 -- 172.21.15.3:0/2374028866 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fa524010430 con 0x7fa5281029b0 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e1e10 0x5562a49e21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 0x5562a49f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49f1500 0x5562a49f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5562a49475d0 con 0x5562a49e2730 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5562a4900890 con 0x5562a49e1e10 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa53342c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5562a4943590 con 0x5562a49f1500 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 0x5562a49f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.756 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 0x5562a49f0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52178/0 (socket says 172.21.15.3:52178) 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2010483585 learned_addr learned my addr 172.21.15.3:0/2010483585 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e1e10 0x5562a49e21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49f1500 0x5562a49f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2010483585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49f1500 msgr2=0x5562a49f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49f1500 0x5562a49f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2010483585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e1e10 msgr2=0x5562a49e21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e1e10 0x5562a49e21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.749+0000 7fa52e4b0640 1 -- 172.21.15.3:0/2010483585 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a4659210 con 0x5562a49e2730 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e1e10 0x5562a49e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa51f7fe640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa5140b2a70 0x7fa5140b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 0x5562a49f0fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa5240e0180 tx=0x7fa5240eb640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0ea91086c101c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa5140b2a70 0x7fa5140b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.757 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa51cff9640 1 -- 172.21.15.3:0/2010483585 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240a5b60 con 0x5562a49e2730 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa51f7fe640 1 -- 172.21.15.3:0/2374028866 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa5240e2f80 con 0x7fa5281029b0 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa51cff9640 1 -- 172.21.15.3:0/2010483585 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5240ebe40 con 0x5562a49e2730 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.750+0000 7fa51cff9640 1 -- 172.21.15.3:0/2010483585 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240a9410 con 0x5562a49e2730 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 -- 172.21.15.3:0/2010483585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 msgr2=0x5562a49f0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa5140b2a70 0x7fa5140b4f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa520004af0 tx=0x7fa520005c50 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 0x5562a49f0fc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa5240e0180 tx=0x7fa5240eb640 comp rx=0 tx=0).stop 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 -- 172.21.15.3:0/2010483585 shutdown_connections 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49f1500 0x5562a49f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e2730 0x5562a49f0fc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 --2- 172.21.15.3:0/2010483585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e1e10 0x5562a49e21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 -- 172.21.15.3:0/2010483585 >> 172.21.15.3:0/2010483585 conn(0x5562a49de420 msgr2=0x5562a49de800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 -- 172.21.15.3:0/2010483585 shutdown_connections 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 -- 172.21.15.3:0/2010483585 wait complete. 2024-05-08T05:54:29.758 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 -- start start 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.751+0000 7fa53342c740 1 Processor -- start 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 0x5562a49ee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e2730 0x5562a49ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49f1500 0x5562a49eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5562a49f4130 con 0x5562a49e1e10 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5562a4904370 con 0x5562a49f1500 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5562a49f42b0 con 0x5562a49e2730 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa53342c740 10 client.0 _ll_register_callbacks cb 0x5562a4658a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 0x5562a49ee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 0x5562a49ee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52180/0 (socket says 172.21.15.3:52180) 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 -- 172.21.15.3:0/1599734678 learned_addr learned my addr 172.21.15.3:0/1599734678 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e2730 0x5562a49ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49f1500 0x5562a49eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.759 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 -- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e2730 msgr2=0x5562a49ee8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e2730 0x5562a49ee8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 -- 172.21.15.3:0/1599734678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49f1500 msgr2=0x5562a49eee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49f1500 0x5562a49eee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 -- 172.21.15.3:0/1599734678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a4725620 con 0x5562a49e1e10 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 0x5562a49ee3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa5240a9c30 tx=0x7fa524009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bb562e9b437b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.752+0000 7fa511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa511ffb640 1 -- 172.21.15.3:0/1599734678 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240a95f0 con 0x5562a49e1e10 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa511ffb640 1 -- 172.21.15.3:0/1599734678 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa524010a90 con 0x5562a49e1e10 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562a49551e0 con 0x5562a49e1e10 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa511ffb640 1 -- 172.21.15.3:0/1599734678 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa5240a9a70 con 0x5562a49e1e10 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 10 client.15339 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:29.760 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5562a49f4130 con 0x5562a49e1e10 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa511ffb640 1 -- 172.21.15.3:0/1599734678 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5562a49551e0 con 0x5562a49e1e10 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa511ffb640 1 -- 172.21.15.3:0/1599734678 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fa5240c4120 con 0x5562a49e1e10 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 10 client.15339 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 10 client.15339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 20 client.15339 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a01150 0x5562a4a03540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 4 client.15339 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562a494f050 con 0x5562a4a01150 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.753+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a01150 0x5562a4a03540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.754+0000 7fa511ffb640 10 client.15339 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.754+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a01150 0x5562a4a03540 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3062052fa4fd8328 server_cookie=f19d41bd501cc340 in_seq=0 out_seq=0 2024-05-08T05:54:29.761 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa511ffb640 1 -- 172.21.15.3:0/1599734678 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562a494f050 con 0x5562a4a01150 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa511ffb640 10 client.15339 handle_command_reply: tid=0 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 client.15339 shutdown 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 msgr2=0x5562a49ee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 0x5562a49ee3a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa5240a9c30 tx=0x7fa524009700 comp rx=0 tx=0).stop 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 shutdown_connections 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a01150 0x5562a4a03540 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49f1500 0x5562a49eee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e2730 0x5562a49ee8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa511ffb640 0 client.15339 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 --2- 172.21.15.3:0/1599734678 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49e1e10 0x5562a49ee3a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 >> 172.21.15.3:0/1599734678 conn(0x5562a49de420 msgr2=0x5562a49e4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:29.762 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 shutdown_connections 2024-05-08T05:54:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 1 -- 172.21.15.3:0/1599734678 wait complete. 2024-05-08T05:54:29.763 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.755+0000 7fa53342c740 20 client.15339 trim_cache size 0 max 16384 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.770+0000 7fa53342c740 1 Processor -- start 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.770+0000 7fa53342c740 1 -- start start 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa53342c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa53342c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a4a08cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa53342c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a4a09210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa53342c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281abb00 con 0x5562a49f1500 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa53342c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ae3a0 con 0x5562a49e2730 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa53342c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ac190 con 0x5562a49e1e10 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a4a09210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.777 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a4a09210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52184/0 (socket says 172.21.15.3:52184) 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 -- 172.21.15.3:0/2664392650 learned_addr learned my addr 172.21.15.3:0/2664392650 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a4a08cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 -- 172.21.15.3:0/2664392650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 msgr2=0x5562a49ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 -- 172.21.15.3:0/2664392650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 msgr2=0x5562a4a08cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a4a08cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 -- 172.21.15.3:0/2664392650 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562a49e2c70 con 0x5562a49f1500 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a4a08cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a4a09210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa520035000 tx=0x7fa520040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0fdb2223bb8321 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa513fff640 1 -- 172.21.15.3:0/2664392650 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520056070 con 0x5562a49f1500 2024-05-08T05:54:29.778 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.771+0000 7fa513fff640 1 -- 172.21.15.3:0/2664392650 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa520036040 con 0x5562a49f1500 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa513fff640 1 -- 172.21.15.3:0/2664392650 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520047620 con 0x5562a49f1500 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 -- 172.21.15.3:0/2664392650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 msgr2=0x5562a4a09210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a4a09210 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa520035000 tx=0x7fa520040000 comp rx=0 tx=0).stop 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 -- 172.21.15.3:0/2664392650 shutdown_connections 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a4a09210 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a4a08cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 --2- 172.21.15.3:0/2664392650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 -- 172.21.15.3:0/2664392650 >> 172.21.15.3:0/2664392650 conn(0x5562a49de420 msgr2=0x5562a49de800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 -- 172.21.15.3:0/2664392650 shutdown_connections 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 -- 172.21.15.3:0/2664392650 wait complete. 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 -- start start 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.772+0000 7fa53342c740 1 Processor -- start 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a49eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a49ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281ac860 con 0x5562a49f1500 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281adcd0 con 0x5562a49e2730 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281acf30 con 0x5562a49e1e10 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa53342c740 10 client.0 _ll_register_callbacks cb 0x7fa5240caf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a49ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a49ebf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:52186/0 (socket says 172.21.15.3:52186) 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 -- 172.21.15.3:0/3127282120 learned_addr learned my addr 172.21.15.3:0/3127282120 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52dcaf640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a49eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 -- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 msgr2=0x5562a49eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 -- 172.21.15.3:0/3127282120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 msgr2=0x5562a49eb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:29.780 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a49eb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 -- 172.21.15.3:0/3127282120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562a4a6aba0 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa52ecb1640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a49ebf50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa520005cc0 tx=0x7fa520030a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd803b5aa5aa8a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.773+0000 7fa511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa511ffb640 1 -- 172.21.15.3:0/3127282120 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520056070 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa520036040 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa511ffb640 1 -- 172.21.15.3:0/3127282120 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa5200475a0 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa511ffb640 1 -- 172.21.15.3:0/3127282120 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa520030da0 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 10 client.15345 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5562a4a6be50 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa511ffb640 1 -- 172.21.15.3:0/3127282120 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa520036040 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa511ffb640 1 -- 172.21.15.3:0/3127282120 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fa520030070 con 0x5562a49f1500 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 10 client.15345 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:29.781 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 20 client.15345 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a6c520 0x5562a4a6e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 4 client.15345 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.774+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562a48ca5f0 con 0x5562a4a6c520 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.775+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a6c520 0x5562a4a6e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.775+0000 7fa511ffb640 10 client.15345 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:29.782 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.775+0000 7fa52e4b0640 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a6c520 0x5562a4a6e940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a28fc89e913bcd41 server_cookie=8f7168b2ff5e3b6c in_seq=0 out_seq=0 2024-05-08T05:54:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa511ffb640 1 -- 172.21.15.3:0/3127282120 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562a48ca5f0 con 0x5562a4a6c520 2024-05-08T05:54:29.783 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa511ffb640 10 client.15345 handle_command_reply: tid=0 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 client.15345 shutdown 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 msgr2=0x5562a49ebf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a49ebf50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa520005cc0 tx=0x7fa520030a20 comp rx=0 tx=0).stop 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 shutdown_connections 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5562a4a6c520 0x5562a4a6e940 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5562a49f1500 0x5562a49ebf50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562a49e2730 0x5562a49eb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa511ffb640 0 client.15345 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 --2- 172.21.15.3:0/3127282120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5562a49e1e10 0x5562a49eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 >> 172.21.15.3:0/3127282120 conn(0x5562a49de420 msgr2=0x5562a49ee340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 shutdown_connections 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 1 -- 172.21.15.3:0/3127282120 wait complete. 2024-05-08T05:54:29.784 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.779+0000 7fa53342c740 20 client.15345 trim_cache size 0 max 16384 2024-05-08T05:54:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:54:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:54:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:54:29.785 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa5140b2a70 msgr2=0x7fa5140b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa5140b2a70 0x7fa5140b4f30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa520004af0 tx=0x7fa520005c50 comp rx=0 tx=0).stop 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 msgr2=0x7fa5281991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 0x7fa5281991c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa52400e9f0 tx=0x7fa52400eeb0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 shutdown_connections 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa5140b2a70 0x7fa5140b4f30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa5281089c0 0x7fa52819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.786 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa528103370 0x7fa528199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 --2- 172.21.15.3:0/2374028866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa5281029b0 0x7fa5281991c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 >> 172.21.15.3:0/2374028866 conn(0x7fa5280fe4a0 msgr2=0x7fa5280fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 shutdown_connections 2024-05-08T05:54:29.787 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:29.780+0000 7fa530714640 1 -- 172.21.15.3:0/2374028866 wait complete. 2024-05-08T05:54:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:29 smithi003 ceph-mon[30137]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-08T05:54:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:29 smithi089 ceph-mon[34495]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-08T05:54:30.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:29 smithi104 ceph-mon[34559]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-08T05:54:31.836 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-08T05:54:31.836 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-05-08T05:54:31.836 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:54:31.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:31 smithi003 ceph-mon[30137]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:54:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:31 smithi089 ceph-mon[34495]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:54:32.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:31 smithi104 ceph-mon[34559]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:54:32.107 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:33.876 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:33 smithi003 ceph-mon[30137]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 -- 172.21.15.3:0/4172941565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc06bb40 msgr2=0x7fbcdc110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/4172941565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc110330 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400b0a0 tx=0x7fbcd400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 -- 172.21.15.3:0/4172941565 shutdown_connections 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/4172941565 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc110330 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/4172941565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc109510 0x7fbcdc1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/4172941565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcdc06b180 0x7fbcdc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:33.988 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.987+0000 7fbce0bc8700 1 -- 172.21.15.3:0/4172941565 >> 172.21.15.3:0/4172941565 conn(0x7fbcdc079350 msgr2=0x7fbcdc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:33.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.988+0000 7fbce0bc8700 1 -- 172.21.15.3:0/4172941565 shutdown_connections 2024-05-08T05:54:33.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.988+0000 7fbce0bc8700 1 -- 172.21.15.3:0/4172941565 wait complete. 2024-05-08T05:54:33.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.988+0000 7fbce0bc8700 1 Processor -- start 2024-05-08T05:54:33.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.988+0000 7fbce0bc8700 1 -- start start 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbce0bc8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcdc06b180 0x7fbcdc19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbce0bc8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbce0bc8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 0x7fbcdc199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbce0bc8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc113620 con 0x7fbcdc109510 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbce0bc8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1134a0 con 0x7fbcdc06bb40 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbce0bc8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc1137a0 con 0x7fbcdc06b180 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcdad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 0x7fbcdc199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcdad9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 0x7fbcdc199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42496/0 (socket says 172.21.15.3:42496) 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcdad9d700 1 -- 172.21.15.3:0/3603539011 learned_addr learned my addr 172.21.15.3:0/3603539011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcda59c700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcdc06b180 0x7fbcdc19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 -- 172.21.15.3:0/3603539011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcdc06b180 msgr2=0x7fbcdc19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:33.991 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcdc06b180 0x7fbcdc19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 -- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 msgr2=0x7fbcdc199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 0x7fbcdc199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 -- 172.21.15.3:0/3603539011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc19a060 con 0x7fbcdc06bb40 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcdad9d700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 0x7fbcdc199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.989+0000 7fbcd9d9b700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc19fc80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000d950 tx=0x7fbcd000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.990+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0016070 con 0x7fbcdc06bb40 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.990+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcd000fd40 con 0x7fbcdc06bb40 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.990+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd000e3b0 con 0x7fbcdc06bb40 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.990+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc19a350 con 0x7fbcdc06bb40 2024-05-08T05:54:33.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.990+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc1a6610 con 0x7fbcdc06bb40 2024-05-08T05:54:33.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.991+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc06a190 con 0x7fbcdc06bb40 2024-05-08T05:54:33.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.992+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fbcd0019070 con 0x7fbcdc06bb40 2024-05-08T05:54:33.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.993+0000 7fbccb7fe700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fbcc40b2b10 0x7fbcc40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:33.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.993+0000 7fbcda59c700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fbcc40b2b10 0x7fbcc40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:33.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.993+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbcd00e2b50 con 0x7fbcdc06bb40 2024-05-08T05:54:33.994 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.994+0000 7fbcda59c700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fbcc40b2b10 0x7fbcc40b4fd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc19aa40 tx=0x7fbccc0073d0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:33.996 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:33.997+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbcd00a9f30 con 0x7fbcdc06bb40 2024-05-08T05:54:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:33 smithi089 ceph-mon[34495]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:34.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:33 smithi104 ceph-mon[34559]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.267+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcdc02e510 con 0x7fbcdc06bb40 2024-05-08T05:54:34.268 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.268+0000 7fbccb7fe700 1 -- 172.21.15.3:0/3603539011 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7fbcd00a9830 con 0x7fbcdc06bb40 2024-05-08T05:54:34.269 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:54:34.269 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fbcc40b2b10 msgr2=0x7fbcc40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fbcc40b2b10 0x7fbcc40b4fd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc19aa40 tx=0x7fbccc0073d0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc06bb40 msgr2=0x7fbcdc19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc19fc80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000d950 tx=0x7fbcd000dc60 comp rx=0 tx=0).stop 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 shutdown_connections 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fbcdc109510 0x7fbcdc199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fbcc40b2b10 0x7fbcc40b4fd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcdc06b180 0x7fbcdc19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 --2- 172.21.15.3:0/3603539011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbcdc06bb40 0x7fbcdc19fc80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:34.271 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 >> 172.21.15.3:0/3603539011 conn(0x7fbcdc079350 msgr2=0x7fbcdc101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 shutdown_connections 2024-05-08T05:54:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:34.271+0000 7fbce0bc8700 1 -- 172.21.15.3:0/3603539011 wait complete. 2024-05-08T05:54:34.272 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T05:54:34.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:34 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3603539011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:35.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:34 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3603539011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:35.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:34 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3603539011' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:54:35.034 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24493 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0' 2024-05-08T05:54:35.034 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24493 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0 2024-05-08T05:54:35.293 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:35 smithi089 ceph-mon[34495]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:36.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:35 smithi104 ceph-mon[34559]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:36.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:35 smithi003 ceph-mon[30137]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.006+0000 7faa237fe700 1 -- 172.21.15.3:0/32694038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c108480 msgr2=0x7faa1c108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.006+0000 7faa237fe700 1 --2- 172.21.15.3:0/32694038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c108480 0x7faa1c108860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faa18009990 tx=0x7faa18009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.006+0000 7faa237fe700 1 -- 172.21.15.3:0/32694038 shutdown_connections 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.006+0000 7faa237fe700 1 --2- 172.21.15.3:0/32694038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa1c102d90 0x7faa1c10f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.006+0000 7faa237fe700 1 --2- 172.21.15.3:0/32694038 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c108480 0x7faa1c108860 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 --2- 172.21.15.3:0/32694038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa1c1023d0 0x7faa1c102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 -- 172.21.15.3:0/32694038 >> 172.21.15.3:0/32694038 conn(0x7faa1c0793e0 msgr2=0x7faa1c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 -- 172.21.15.3:0/32694038 shutdown_connections 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 -- 172.21.15.3:0/32694038 wait complete. 2024-05-08T05:54:37.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 Processor -- start 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 -- start start 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.007+0000 7faa237fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa1c1023d0 0x7faa1c19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa237fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 0x7faa1c19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa237fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa1c108480 0x7faa1c1a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa237fe700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c112570 con 0x7faa1c102d90 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa237fe700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1123f0 con 0x7faa1c1023d0 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa237fe700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1126f0 con 0x7faa1c108480 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 0x7faa1c19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 0x7faa1c19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42522/0 (socket says 172.21.15.3:42522) 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 -- 172.21.15.3:0/2961668573 learned_addr learned my addr 172.21.15.3:0/2961668573 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa2159a700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa1c1023d0 0x7faa1c19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa21d9b700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa1c108480 0x7faa1c1a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 -- 172.21.15.3:0/2961668573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa1c108480 msgr2=0x7faa1c1a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa1c108480 0x7faa1c1a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 -- 172.21.15.3:0/2961668573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa1c1023d0 msgr2=0x7faa1c19a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa1c1023d0 0x7faa1c19a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 -- 172.21.15.3:0/2961668573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c1a5e10 con 0x7faa1c102d90 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa20d99700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 0x7faa1c19ad90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9c0 tx=0x7faa0c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa127fc700 1 -- 172.21.15.3:0/2961668573 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00cbd0 con 0x7faa1c102d90 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c1a5fe0 con 0x7faa1c102d90 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa127fc700 1 -- 172.21.15.3:0/2961668573 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa0c00ee60 con 0x7faa1c102d90 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.008+0000 7faa127fc700 1 -- 172.21.15.3:0/2961668573 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c0055c0 con 0x7faa1c102d90 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.009+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c1a6490 con 0x7faa1c102d90 2024-05-08T05:54:37.011 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.011+0000 7faa127fc700 1 -- 172.21.15.3:0/2961668573 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7faa0c004750 con 0x7faa1c102d90 2024-05-08T05:54:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.012+0000 7faa127fc700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7faa080b2bc0 0x7faa080b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.012+0000 7faa2159a700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7faa080b2bc0 0x7faa080b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.012+0000 7faa127fc700 1 -- 172.21.15.3:0/2961668573 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7faa0c0e2ea0 con 0x7faa1c102d90 2024-05-08T05:54:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2159a700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7faa080b2bc0 0x7faa080b5080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faa1800b2e0 tx=0x7faa18005960 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 Processor -- start 2024-05-08T05:54:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 -- start start 2024-05-08T05:54:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaeb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5642bda3c9f0 con 0x5642bdaeae50 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5642bd9db950 con 0x5642bdaef950 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2e091b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642bda389b0 con 0x5642bdaef030 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa21d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaeb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa21d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaeb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42534/0 (socket says 172.21.15.3:42534) 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa21d9b700 1 -- 172.21.15.3:0/1877874280 learned_addr learned my addr 172.21.15.3:0/1877874280 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa2159a700 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.013+0000 7faa20d99700 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa21d9b700 1 -- 172.21.15.3:0/1877874280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 msgr2=0x5642bdaef410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa21d9b700 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaef410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa21d9b700 1 -- 172.21.15.3:0/1877874280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 msgr2=0x5642bdaea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa21d9b700 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa21d9b700 1 -- 172.21.15.3:0/1877874280 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642bd989a60 con 0x5642bdaeae50 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa21d9b700 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaeb2b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faa1400f410 tx=0x7faa1400f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45b099d38402cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa077fe700 1 -- 172.21.15.3:0/1877874280 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14018030 con 0x5642bdaeae50 2024-05-08T05:54:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa077fe700 1 -- 172.21.15.3:0/1877874280 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa14004bb0 con 0x5642bdaeae50 2024-05-08T05:54:37.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.014+0000 7faa077fe700 1 -- 172.21.15.3:0/1877874280 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14005020 con 0x5642bdaeae50 2024-05-08T05:54:37.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 -- 172.21.15.3:0/1877874280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 msgr2=0x5642bdaeb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaeb2b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faa1400f410 tx=0x7faa1400f720 comp rx=0 tx=0).stop 2024-05-08T05:54:37.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 -- 172.21.15.3:0/1877874280 shutdown_connections 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaeb2b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaef410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1877874280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 -- 172.21.15.3:0/1877874280 >> 172.21.15.3:0/1877874280 conn(0x5642bda16290 msgr2=0x5642bdae5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 -- 172.21.15.3:0/1877874280 shutdown_connections 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 -- 172.21.15.3:0/1877874280 wait complete. 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 -- start start 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.015+0000 7faa2e091b80 1 Processor -- start 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaf6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5642bdaf7520 con 0x5642bdaeae50 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5642bd9a2dc0 con 0x5642bdaef950 2024-05-08T05:54:37.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642bdaf76a0 con 0x5642bdaef030 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2e091b80 10 client.0 _ll_register_callbacks cb 0x5642bd9d4ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaf6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaf6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42550/0 (socket says 172.21.15.3:42550) 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 -- 172.21.15.3:0/245564925 learned_addr learned my addr 172.21.15.3:0/245564925 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa20d99700 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 -- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 msgr2=0x5642bdaf39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 -- 172.21.15.3:0/245564925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 msgr2=0x5642bdaf3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 -- 172.21.15.3:0/245564925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642bd9d8820 con 0x5642bdaeae50 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa2159a700 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaf6630 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faa18009990 tx=0x7faa18009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75d97728590bb96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.016+0000 7faa04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa04ff9700 1 -- 172.21.15.3:0/245564925 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18035060 con 0x5642bdaeae50 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa04ff9700 1 -- 172.21.15.3:0/245564925 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7faa1800f460 con 0x5642bdaeae50 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa14004bb0 con 0x5642bdaeae50 2024-05-08T05:54:37.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa04ff9700 1 -- 172.21.15.3:0/245564925 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7faa18005690 con 0x5642bdaeae50 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 10 client.15363 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5642bdb01c60 con 0x5642bdaeae50 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa04ff9700 1 -- 172.21.15.3:0/245564925 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa14004bb0 con 0x5642bdaeae50 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa04ff9700 1 -- 172.21.15.3:0/245564925 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7faa18026790 con 0x5642bdaeae50 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 10 client.15363 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 20 client.15363 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb042a0 0x5642bdb066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 4 client.15363 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.017+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642bdab67e0 con 0x5642bdb042a0 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.018+0000 7faa20d99700 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb042a0 0x5642bdb066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.018+0000 7faa20d99700 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb042a0 0x5642bdb066b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3c362a4f20dc61 server_cookie=c915a0603154da80 in_seq=0 out_seq=0 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.018+0000 7faa04ff9700 10 client.15363 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa04ff9700 1 -- 172.21.15.3:0/245564925 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5642bdab67e0 con 0x5642bdb042a0 2024-05-08T05:54:37.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa04ff9700 10 client.15363 handle_command_reply: tid=0 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 client.15363 shutdown 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 msgr2=0x5642bdaf6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaf6630 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faa18009990 tx=0x7faa18009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 shutdown_connections 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb042a0 0x5642bdb066b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaf6630 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 --2- 172.21.15.3:0/245564925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 >> 172.21.15.3:0/245564925 conn(0x5642bda16290 msgr2=0x5642bdae3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa04ff9700 0 client.15363 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 shutdown_connections 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 1 -- 172.21.15.3:0/245564925 wait complete. 2024-05-08T05:54:37.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.019+0000 7faa2e091b80 20 client.15363 trim_cache size 0 max 16384 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.039+0000 7faa2e091b80 1 Processor -- start 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 -- start start 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdae4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1ad260 con 0x5642bdaeae50 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1afb00 con 0x5642bdaef950 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2e091b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1ad8f0 con 0x5642bdaef030 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdae4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdae4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42564/0 (socket says 172.21.15.3:42564) 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 -- 172.21.15.3:0/794363138 learned_addr learned my addr 172.21.15.3:0/794363138 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:37.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa20d99700 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa21d9b700 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 -- 172.21.15.3:0/794363138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 msgr2=0x5642bdaf3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 -- 172.21.15.3:0/794363138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 msgr2=0x5642bdaf37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.040+0000 7faa2159a700 1 -- 172.21.15.3:0/794363138 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642bdadffd0 con 0x5642bdaeae50 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2159a700 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdae4540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faa18009960 tx=0x7faa18005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8e3ea2df2c4795 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa06ffd700 1 -- 172.21.15.3:0/794363138 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18045070 con 0x5642bdaeae50 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa06ffd700 1 -- 172.21.15.3:0/794363138 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa18005400 con 0x5642bdaeae50 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa06ffd700 1 -- 172.21.15.3:0/794363138 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18035040 con 0x5642bdaeae50 2024-05-08T05:54:37.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 -- 172.21.15.3:0/794363138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 msgr2=0x5642bdae4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdae4540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faa18009960 tx=0x7faa18005d20 comp rx=0 tx=0).stop 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 -- 172.21.15.3:0/794363138 shutdown_connections 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef950 0x5642bdaf37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdae4540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 --2- 172.21.15.3:0/794363138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef030 0x5642bdaf3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.041+0000 7faa2e091b80 1 -- 172.21.15.3:0/794363138 >> 172.21.15.3:0/794363138 conn(0x5642bda16290 msgr2=0x5642bdae3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 -- 172.21.15.3:0/794363138 shutdown_connections 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 -- 172.21.15.3:0/794363138 wait complete. 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 -- start start 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 Processor -- start 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaefd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef030 0x5642bdaf02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef950 0x5642bdaf07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1adfc0 con 0x5642bdaeae50 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1af430 con 0x5642bdaef030 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1ae690 con 0x5642bdaef950 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.042+0000 7faa2e091b80 10 client.0 _ll_register_callbacks cb 0x5642bd9d4ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaefd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaefd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:42572/0 (socket says 172.21.15.3:42572) 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 -- 172.21.15.3:0/1061432052 learned_addr learned my addr 172.21.15.3:0/1061432052 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa20d99700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef030 0x5642bdaf02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa21d9b700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef950 0x5642bdaf07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 -- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef950 msgr2=0x5642bdaf07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef950 0x5642bdaf07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 -- 172.21.15.3:0/1061432052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef030 msgr2=0x5642bdaf02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef030 0x5642bdaf02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 -- 172.21.15.3:0/1061432052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642bdaeec80 con 0x5642bdaeae50 2024-05-08T05:54:37.047 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2159a700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaefd70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faa18005850 tx=0x7faa180247c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6030f652dc62bbb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa04ff9700 1 -- 172.21.15.3:0/1061432052 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa18045070 con 0x5642bdaeae50 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa18005400 con 0x5642bdaeae50 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.043+0000 7faa04ff9700 1 -- 172.21.15.3:0/1061432052 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7faa18026840 con 0x5642bdaeae50 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa04ff9700 1 -- 172.21.15.3:0/1061432052 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7faa18035040 con 0x5642bdaeae50 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 10 client.15369 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5642bdb73e10 con 0x5642bdaeae50 2024-05-08T05:54:37.048 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa04ff9700 1 -- 172.21.15.3:0/1061432052 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa18005400 con 0x5642bdaeae50 2024-05-08T05:54:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa04ff9700 1 -- 172.21.15.3:0/1061432052 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7faa18040910 con 0x5642bdaeae50 2024-05-08T05:54:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 10 client.15369 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:54:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 20 client.15369 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:37.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb74330 0x5642bdb76720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:37.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 4 client.15369 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}] 2024-05-08T05:54:37.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}) -- 0x5642bda4a3d0 con 0x5642bdb74330 2024-05-08T05:54:37.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.044+0000 7faa20d99700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb74330 0x5642bdb76720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:37.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.045+0000 7faa20d99700 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb74330 0x5642bdb76720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a32607e99f1a0b5 server_cookie=db3c965860763074 in_seq=0 out_seq=0 2024-05-08T05:54:37.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.045+0000 7faa04ff9700 10 client.15369 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.054+0000 7faa04ff9700 1 -- 172.21.15.3:0/1061432052 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5642bda4a3d0 con 0x5642bdb74330 2024-05-08T05:54:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.054+0000 7faa04ff9700 10 client.15369 handle_command_reply: tid=0 2024-05-08T05:54:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.054+0000 7faa2e091b80 1 client.15369 shutdown 2024-05-08T05:54:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 msgr2=0x5642bdaefd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaefd70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faa18005850 tx=0x7faa180247c0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 shutdown_connections 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642bdaef950 0x5642bdaf07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5642bdaeae50 0x5642bdaefd70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5642bdb74330 0x5642bdb76720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 --2- 172.21.15.3:0/1061432052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5642bdaef030 0x5642bdaf02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 >> 172.21.15.3:0/1061432052 conn(0x5642bda16290 msgr2=0x5642bdaee500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa04ff9700 0 client.15369 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 shutdown_connections 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 1 -- 172.21.15.3:0/1061432052 wait complete. 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa2e091b80 20 client.15369 trim_cache size 0 max 16384 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:54:37.056 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 23, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24493, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 2, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "b04c9cab": { 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 2, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 60.0, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 79.4, 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-05-08T05:54:37.057 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:54:37.058 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7faa080b2bc0 msgr2=0x7faa080b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa237fe700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7faa080b2bc0 0x7faa080b5080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faa1800b2e0 tx=0x7faa18005960 comp rx=0 tx=0).stop 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.055+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 msgr2=0x7faa1c19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 0x7faa1c19ad90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00e9c0 tx=0x7faa0c00ecd0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 shutdown_connections 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7faa080b2bc0 0x7faa080b5080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa1c108480 0x7faa1c1a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7faa1c1023d0 0x7faa1c19a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 --2- 172.21.15.3:0/2961668573 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7faa1c102d90 0x7faa1c19ad90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:37.059 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 >> 172.21.15.3:0/2961668573 conn(0x7faa1c0793e0 msgr2=0x7faa1c100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 shutdown_connections 2024-05-08T05:54:37.060 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:37.056+0000 7faa237fe700 1 -- 172.21.15.3:0/2961668573 wait complete. 2024-05-08T05:54:37.750 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:37 smithi003 ceph-mon[30137]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:37.782 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b04c9cab', quiesce took 5.9 seconds. Will release after: 19.4 2024-05-08T05:54:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:37 smithi089 ceph-mon[34495]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:37 smithi104 ceph-mon[34559]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:54:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:39 smithi089 ceph-mon[34495]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-08T05:54:40.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:39 smithi104 ceph-mon[34559]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-08T05:54:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:39 smithi003 ceph-mon[30137]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 938 B/s wr, 2 op/s 2024-05-08T05:54:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:41 smithi089 ceph-mon[34495]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2024-05-08T05:54:42.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:41 smithi104 ceph-mon[34559]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2024-05-08T05:54:42.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:41 smithi003 ceph-mon[30137]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 938 B/s wr, 2 op/s 2024-05-08T05:54:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 -- 172.21.15.3:0/4228380974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 msgr2=0x7f8a50108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 --2- 172.21.15.3:0/4228380974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a50108da0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009930 tx=0x7f8a4c02a420 comp rx=0 tx=0).stop 2024-05-08T05:54:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 -- 172.21.15.3:0/4228380974 shutdown_connections 2024-05-08T05:54:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 --2- 172.21.15.3:0/4228380974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a50103370 0x7f8a5010f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.884 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 --2- 172.21.15.3:0/4228380974 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a501029b0 0x7f8a50102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 --2- 172.21.15.3:0/4228380974 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a50108da0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 -- 172.21.15.3:0/4228380974 >> 172.21.15.3:0/4228380974 conn(0x7f8a500fe4a0 msgr2=0x7f8a501008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 -- 172.21.15.3:0/4228380974 shutdown_connections 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 -- 172.21.15.3:0/4228380974 wait complete. 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 Processor -- start 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.882+0000 7f8a58409640 1 -- start start 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a58409640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a501029b0 0x7f8a50199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a58409640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a50103370 0x7f8a50199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a58409640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a5019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a58409640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112c80 con 0x7f8a501089c0 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a58409640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112b00 con 0x7f8a501029b0 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a58409640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a50112e00 con 0x7f8a50103370 2024-05-08T05:54:42.885 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a5019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a561a5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a501029b0 0x7f8a50199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a5019dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53382/0 (socket says 172.21.15.3:53382) 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 -- 172.21.15.3:0/1956271441 learned_addr learned my addr 172.21.15.3:0/1956271441 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a559a4640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a50103370 0x7f8a50199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 -- 172.21.15.3:0/1956271441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a50103370 msgr2=0x7f8a50199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a50103370 0x7f8a50199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 -- 172.21.15.3:0/1956271441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a501029b0 msgr2=0x7f8a50199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a501029b0 0x7f8a50199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 -- 172.21.15.3:0/1956271441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5019e1f0 con 0x7f8a501089c0 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.883+0000 7f8a569a6640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a5019dac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800eb10 tx=0x7f8a4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:42.886 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.884+0000 7f8a477fe640 1 -- 172.21.15.3:0/1956271441 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48004b20 con 0x7f8a501089c0 2024-05-08T05:54:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.884+0000 7f8a477fe640 1 -- 172.21.15.3:0/1956271441 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a48004cc0 con 0x7f8a501089c0 2024-05-08T05:54:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.884+0000 7f8a477fe640 1 -- 172.21.15.3:0/1956271441 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a48010670 con 0x7f8a501089c0 2024-05-08T05:54:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.884+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5019e4e0 con 0x7f8a501089c0 2024-05-08T05:54:42.887 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.884+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a501a5d70 con 0x7f8a501089c0 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.886+0000 7f8a477fe640 1 -- 172.21.15.3:0/1956271441 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f8a48005020 con 0x7f8a501089c0 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 Processor -- start 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 -- start start 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced05302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 0x55ced053abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced053b0f0 0x55ced053d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ced04955d0 con 0x55ced0530830 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ced044e890 con 0x55ced052ff10 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ced0491590 con 0x55ced053b0f0 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.887+0000 7f8a561a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 0x55ced053abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 0x55ced053abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53390/0 (socket says 172.21.15.3:53390) 2024-05-08T05:54:42.890 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 -- 172.21.15.3:0/4224819628 learned_addr learned my addr 172.21.15.3:0/4224819628 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a559a4640 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced053b0f0 0x55ced053d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a559a4640 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced05302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 -- 172.21.15.3:0/4224819628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced053b0f0 msgr2=0x55ced053d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced053b0f0 0x55ced053d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 -- 172.21.15.3:0/4224819628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 msgr2=0x55ced05302f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced05302f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 -- 172.21.15.3:0/4224819628 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced01a7210 con 0x55ced0530830 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a477fe640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8a3c0b2af0 0x7f8a3c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a561a5640 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 0x55ced053abb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c02a940 tx=0x7f8a4c031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8141855de4582cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a559a4640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8a3c0b2af0 0x7f8a3c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a477fe640 1 -- 172.21.15.3:0/1956271441 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8a480e3150 con 0x7f8a501089c0 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a44bfe640 1 -- 172.21.15.3:0/4224819628 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c040070 con 0x55ced0530830 2024-05-08T05:54:42.891 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a44bfe640 1 -- 172.21.15.3:0/4224819628 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4c0316a0 con 0x55ced0530830 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.888+0000 7f8a44bfe640 1 -- 172.21.15.3:0/4224819628 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c035040 con 0x55ced0530830 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/4224819628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 msgr2=0x55ced053abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 0x55ced053abb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c02a940 tx=0x7f8a4c031bc0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a559a4640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8a3c0b2af0 0x7f8a3c0b4fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55ced052d840 tx=0x7f8a40004730 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/4224819628 shutdown_connections 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced053b0f0 0x55ced053d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced0530830 0x55ced053abb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/4224819628 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced05302f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/4224819628 >> 172.21.15.3:0/4224819628 conn(0x55ced052c420 msgr2=0x55ced052c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/4224819628 shutdown_connections 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.889+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/4224819628 wait complete. 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 -- start start 2024-05-08T05:54:42.892 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 Processor -- start 2024-05-08T05:54:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0538000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.896 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0538540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0538a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55ced053dcc0 con 0x55ced053b0f0 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ced0452370 con 0x55ced052ff10 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ced053de40 con 0x55ced0530830 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a5b1ff740 10 client.0 _ll_register_callbacks cb 0x55ced01a6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a569a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0538a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a569a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0538a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53404/0 (socket says 172.21.15.3:53404) 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a569a6640 1 -- 172.21.15.3:0/2070226760 learned_addr learned my addr 172.21.15.3:0/2070226760 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a561a5640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0538000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.890+0000 7f8a559a4640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0538540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a569a6640 1 -- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 msgr2=0x55ced0538540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a569a6640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0538540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.897 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a569a6640 1 -- 172.21.15.3:0/2070226760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 msgr2=0x55ced0538000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a569a6640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0538000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a569a6640 1 -- 172.21.15.3:0/2070226760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced0273620 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a569a6640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0538a80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a480e0350 tx=0x7f8a48005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794cd3c9e0975b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a39ffb640 1 -- 172.21.15.3:0/2070226760 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480a9980 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a39ffb640 1 -- 172.21.15.3:0/2070226760 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8a480b2a50 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c0316a0 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a39ffb640 1 -- 172.21.15.3:0/2070226760 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8a480aa570 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a5b1ff740 10 client.15381 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ced0549bb0 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a39ffb640 1 -- 172.21.15.3:0/2070226760 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a4c0316a0 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.891+0000 7f8a39ffb640 1 -- 172.21.15.3:0/2070226760 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8a480a57e0 con 0x55ced053b0f0 2024-05-08T05:54:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a5b1ff740 10 client.15381 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a5b1ff740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a5b1ff740 20 client.15381 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced054b0c0 0x55ced054d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a5b1ff740 4 client.15381 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ced04a31e0 con 0x55ced054b0c0 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a561a5640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced054b0c0 0x55ced054d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a561a5640 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced054b0c0 0x55ced054d4d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d55a2786ce55222 server_cookie=3de0d0b36dc8f128 in_seq=0 out_seq=0 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.892+0000 7f8a39ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a39ffb640 1 -- 172.21.15.3:0/2070226760 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ced04a31e0 con 0x55ced054b0c0 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a39ffb640 10 client.15381 handle_command_reply: tid=0 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 client.15381 shutdown 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 msgr2=0x55ced0538a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0538a80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8a480e0350 tx=0x7f8a48005610 comp rx=0 tx=0).stop 2024-05-08T05:54:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 shutdown_connections 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced054b0c0 0x55ced054d4d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0538a80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0538540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a39ffb640 0 client.15381 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/2070226760 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0538000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 >> 172.21.15.3:0/2070226760 conn(0x55ced052c420 msgr2=0x55ced05321a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 shutdown_connections 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.893+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/2070226760 wait complete. 2024-05-08T05:54:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.894+0000 7f8a5b1ff740 20 client.15381 trim_cache size 0 max 16384 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 Processor -- start 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 -- start start 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0552de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0553320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0535280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501addd0 con 0x55ced053b0f0 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501b0670 con 0x55ced052ff10 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501ae460 con 0x55ced0530830 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0535280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0535280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53414/0 (socket says 172.21.15.3:53414) 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 -- 172.21.15.3:0/3163131313 learned_addr learned my addr 172.21.15.3:0/3163131313 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a559a4640 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0553320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a561a5640 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0552de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 -- 172.21.15.3:0/3163131313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 msgr2=0x55ced0553320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0553320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 -- 172.21.15.3:0/3163131313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 msgr2=0x55ced0552de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.909+0000 7f8a569a6640 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0552de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a569a6640 1 -- 172.21.15.3:0/3163131313 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced0533be0 con 0x55ced053b0f0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a569a6640 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0535280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a480049d0 tx=0x7f8a480a9d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f38099b135177c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a3bfff640 1 -- 172.21.15.3:0/3163131313 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480a97c0 con 0x55ced053b0f0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a3bfff640 1 -- 172.21.15.3:0/3163131313 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a480a9980 con 0x55ced053b0f0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a3bfff640 1 -- 172.21.15.3:0/3163131313 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a480e0690 con 0x55ced053b0f0 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3163131313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 msgr2=0x55ced0535280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0535280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8a480049d0 tx=0x7f8a480a9d70 comp rx=0 tx=0).stop 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3163131313 shutdown_connections 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced053b0f0 0x55ced0535280 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0553320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3163131313 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced052ff10 0x55ced0552de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3163131313 >> 172.21.15.3:0/3163131313 conn(0x55ced052c420 msgr2=0x55ced052b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3163131313 shutdown_connections 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.910+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3163131313 wait complete. 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 -- start start 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 Processor -- start 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 0x55ced05351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0535710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced053b0f0 0x55ced05b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501aeb30 con 0x55ced052ff10 2024-05-08T05:54:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501affa0 con 0x55ced053b0f0 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8a501af200 con 0x55ced0530830 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a5b1ff740 10 client.0 _ll_register_callbacks cb 0x7f8a480a5de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a561a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 0x55ced05351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a561a5640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 0x55ced05351d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:53424/0 (socket says 172.21.15.3:53424) 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a561a5640 1 -- 172.21.15.3:0/3372587991 learned_addr learned my addr 172.21.15.3:0/3372587991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a559a4640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0535710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.911+0000 7f8a569a6640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced053b0f0 0x55ced05b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a561a5640 1 -- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 msgr2=0x55ced0535710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a561a5640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0535710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a561a5640 1 -- 172.21.15.3:0/3372587991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced053b0f0 msgr2=0x55ced05b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a561a5640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced053b0f0 0x55ced05b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a561a5640 1 -- 172.21.15.3:0/3372587991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced0536010 con 0x55ced052ff10 2024-05-08T05:54:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a561a5640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 0x55ced05351d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005130 tx=0x7f8a4c0050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423a1f56e2d75b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a39ffb640 1 -- 172.21.15.3:0/3372587991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c035040 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a39ffb640 1 -- 172.21.15.3:0/3372587991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8a4c0054a0 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a480a9980 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a39ffb640 1 -- 172.21.15.3:0/3372587991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c05f050 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a5b1ff740 10 client.15387 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55ced05b6010 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a39ffb640 1 -- 172.21.15.3:0/3372587991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a480a9980 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.912+0000 7f8a39ffb640 1 -- 172.21.15.3:0/3372587991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8a4c04d040 con 0x55ced052ff10 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a5b1ff740 10 client.15387 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a5b1ff740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a5b1ff740 20 client.15387 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced05b6350 0x55ced05b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a5b1ff740 4 client.15387 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:54:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ced049d050 con 0x55ced05b6350 2024-05-08T05:54:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a559a4640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced05b6350 0x55ced05b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a39ffb640 10 client.15387 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.913+0000 7f8a559a4640 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced05b6350 0x55ced05b8740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3515ec691050cb1a server_cookie=26deface56cc94b7 in_seq=0 out_seq=0 2024-05-08T05:54:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a39ffb640 1 -- 172.21.15.3:0/3372587991 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ced049d050 con 0x55ced05b6350 2024-05-08T05:54:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a39ffb640 10 client.15387 handle_command_reply: tid=0 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 client.15387 shutdown 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 msgr2=0x55ced05351d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 0x55ced05351d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005130 tx=0x7f8a4c0050c0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 shutdown_connections 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55ced05b6350 0x55ced05b8740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ced053b0f0 0x55ced05b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ced0530830 0x55ced0535710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a39ffb640 0 client.15387 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 --2- 172.21.15.3:0/3372587991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55ced052ff10 0x55ced05351d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 >> 172.21.15.3:0/3372587991 conn(0x55ced052c420 msgr2=0x55ced05318c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 shutdown_connections 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 1 -- 172.21.15.3:0/3372587991 wait complete. 2024-05-08T05:54:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.920+0000 7f8a5b1ff740 20 client.15387 trim_cache size 0 max 16384 2024-05-08T05:54:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:54:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:54:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:54:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8a3c0b2af0 msgr2=0x7f8a3c0b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8a3c0b2af0 0x7f8a3c0b4fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55ced052d840 tx=0x7f8a40004730 comp rx=0 tx=0).stop 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 msgr2=0x7f8a5019dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a5019dac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800eb10 tx=0x7f8a4800c2d0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 shutdown_connections 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8a3c0b2af0 0x7f8a3c0b4fb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8a501089c0 0x7f8a5019dac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8a50103370 0x7f8a50199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 --2- 172.21.15.3:0/1956271441 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8a501029b0 0x7f8a50199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 >> 172.21.15.3:0/1956271441 conn(0x7f8a500fe4a0 msgr2=0x7f8a500fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 shutdown_connections 2024-05-08T05:54:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:42.921+0000 7f8a58409640 1 -- 172.21.15.3:0/1956271441 wait complete. 2024-05-08T05:54:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:43 smithi089 ceph-mon[34495]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-08T05:54:44.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:43 smithi104 ceph-mon[34559]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-08T05:54:44.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:43 smithi003 ceph-mon[30137]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-08T05:54:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:45 smithi089 ceph-mon[34495]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 1 op/s 2024-05-08T05:54:46.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:45 smithi104 ceph-mon[34559]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 1 op/s 2024-05-08T05:54:46.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:45 smithi003 ceph-mon[30137]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 1 op/s 2024-05-08T05:54:48.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:47 smithi089 ceph-mon[34495]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 1 op/s 2024-05-08T05:54:48.086 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:47 smithi104 ceph-mon[34559]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 1 op/s 2024-05-08T05:54:48.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:47 smithi003 ceph-mon[30137]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 682 B/s wr, 1 op/s 2024-05-08T05:54:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:49 smithi089 ceph-mon[34495]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-05-08T05:54:50.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:49 smithi104 ceph-mon[34559]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-05-08T05:54:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:49 smithi003 ceph-mon[30137]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-05-08T05:54:52.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:51 smithi089 ceph-mon[34495]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:52.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:51 smithi104 ceph-mon[34559]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:52.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:51 smithi003 ceph-mon[30137]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:53 smithi089 ceph-mon[34495]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-05-08T05:54:54.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:53 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:54:54.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:53 smithi104 ceph-mon[34559]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-05-08T05:54:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:53 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:54:54.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:53 smithi003 ceph-mon[30137]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-05-08T05:54:54.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:53 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:54:55.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:54 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:54:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:54 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:54:55.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:54 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:54:55.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:54 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:54:55.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:54 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:54:55.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:54 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:54:55.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:54 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:54:55.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:54 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:54:55.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:54 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:54:56.014 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 -- 172.21.15.3:0/3310662552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 msgr2=0x7fb3ec104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/3310662552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec104000 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009900 tx=0x7fb3dc02a480 comp rx=0 tx=0).stop 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 -- 172.21.15.3:0/3310662552 shutdown_connections 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/3310662552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ec10cff0 0x7fb3ec10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/3310662552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3ec104540 0x7fb3ec10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/3310662552 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec104000 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 -- 172.21.15.3:0/3310662552 >> 172.21.15.3:0/3310662552 conn(0x7fb3ec0789d0 msgr2=0x7fb3ec078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 -- 172.21.15.3:0/3310662552 shutdown_connections 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.012+0000 7fb3f09c1640 1 -- 172.21.15.3:0/3310662552 wait complete. 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 Processor -- start 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 -- start start 2024-05-08T05:54:56.015 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ec104540 0x7fb3ec101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3ec10cff0 0x7fb3ec1022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec112c30 con 0x7fb3ec103c20 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec112ab0 con 0x7fb3ec104540 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3f09c1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec112db0 con 0x7fb3ec10cff0 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60628/0 (socket says 172.21.15.3:60628) 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.013+0000 7fb3ea59c640 1 -- 172.21.15.3:0/2085920455 learned_addr learned my addr 172.21.15.3:0/2085920455 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3ec10cff0 0x7fb3ec1022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ec104540 0x7fb3ec101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ea59c640 1 -- 172.21.15.3:0/2085920455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3ec10cff0 msgr2=0x7fb3ec1022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3ec10cff0 0x7fb3ec1022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.016 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ea59c640 1 -- 172.21.15.3:0/2085920455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ec104540 msgr2=0x7fb3ec101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ec104540 0x7fb3ec101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ea59c640 1 -- 172.21.15.3:0/2085920455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ec1a76b0 con 0x7fb3ec103c20 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec101870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc005940 tx=0x7fb3dc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3db7fe640 1 -- 172.21.15.3:0/2085920455 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc03f030 con 0x7fb3ec103c20 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ec1a7940 con 0x7fb3ec103c20 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3db7fe640 1 -- 172.21.15.3:0/2085920455 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3dc003cb0 con 0x7fb3ec103c20 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3db7fe640 1 -- 172.21.15.3:0/2085920455 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc035070 con 0x7fb3ec103c20 2024-05-08T05:54:56.017 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.014+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ec1a7e30 con 0x7fb3ec103c20 2024-05-08T05:54:56.018 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:55 smithi089 ceph-mon[34495]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:56.023 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:55 smithi104 ceph-mon[34559]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.017+0000 7fb3f37b5740 1 Processor -- start 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 -- start start 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3db7fe640 1 -- 172.21.15.3:0/2085920455 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7fb3dc042030 con 0x7fb3ec103c20 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a7770312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77703bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a77703e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a776f965d0 con 0x55a777031830 2024-05-08T05:54:56.025 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a776f4f890 con 0x55a77703c0f0 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a776f92590 con 0x55a777030f10 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77703bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77703bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60640/0 (socket says 172.21.15.3:60640) 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 -- 172.21.15.3:0/3584375428 learned_addr learned my addr 172.21.15.3:0/3584375428 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a77703e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a7770312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 -- 172.21.15.3:0/3584375428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 msgr2=0x55a7770312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a7770312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 -- 172.21.15.3:0/3584375428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 msgr2=0x55a77703e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.026 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a77703e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.018+0000 7fb3ea59c640 1 -- 172.21.15.3:0/3584375428 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a776ca8210 con 0x55a777031830 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77703bbb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc100640 tx=0x7fb3dc109680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7306ab58299d4dab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3d8ff9640 1 -- 172.21.15.3:0/3584375428 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc103870 con 0x55a777031830 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3d8ff9640 1 -- 172.21.15.3:0/3584375428 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3dc110030 con 0x55a777031830 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3d8ff9640 1 -- 172.21.15.3:0/3584375428 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc115640 con 0x55a777031830 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3db7fe640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fb3d40b2a70 0x7fb3d40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fb3d40b2a70 0x7fb3d40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.019+0000 7fb3db7fe640 1 -- 172.21.15.3:0/2085920455 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb3dc103440 con 0x7fb3ec103c20 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fb3d40b2a70 0x7fb3d40b4f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec06b2b0 tx=0x7fb3e40096a0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3584375428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 msgr2=0x55a77703bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77703bbb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc100640 tx=0x7fb3dc109680 comp rx=0 tx=0).stop 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3584375428 shutdown_connections 2024-05-08T05:54:56.027 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a77703e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77703bbb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3584375428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a7770312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3584375428 >> 172.21.15.3:0/3584375428 conn(0x55a77702d420 msgr2=0x55a77702d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3584375428 shutdown_connections 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3584375428 wait complete. 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.020+0000 7fb3f37b5740 1 -- start start 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 Processor -- start 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 0x55a777038fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777031830 0x55a777039520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77703c0f0 0x55a777039a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55a77703eae0 con 0x55a777030f10 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a776f53370 con 0x55a777031830 2024-05-08T05:54:56.028 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a77703ec60 con 0x55a77703c0f0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3f37b5740 10 client.0 _ll_register_callbacks cb 0x55a776ca7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 0x55a777038fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 0x55a777038fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60642/0 (socket says 172.21.15.3:60642) 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 -- 172.21.15.3:0/208429497 learned_addr learned my addr 172.21.15.3:0/208429497 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77703c0f0 0x55a777039a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777031830 0x55a777039520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 -- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77703c0f0 msgr2=0x55a777039a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77703c0f0 0x55a777039a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 -- 172.21.15.3:0/208429497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777031830 msgr2=0x55a777039520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777031830 0x55a777039520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.021+0000 7fb3ea59c640 1 -- 172.21.15.3:0/208429497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a776d74620 con 0x55a777030f10 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 0x55a777038fe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0ca1c0 tx=0x7fb3dc03b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f8e1ac1f6f39e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:56.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/208429497 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc049030 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/208429497 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb3dc1090e0 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3dc110030 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/208429497 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb3dc0c60f0 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3f37b5740 10 client.15399 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a77703eae0 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/208429497 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3dc110030 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/208429497 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fb3dc103120 con 0x55a777030f10 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3f37b5740 10 client.15399 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3f37b5740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.022+0000 7fb3f37b5740 20 client.15399 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.023+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a77704de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.023+0000 7fb3f37b5740 4 client.15399 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:54:56.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.023+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a776fa41e0 con 0x55a77704ba90 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.023+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a77704de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.023+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a77704de80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df08cffbfb0225d server_cookie=b4d9e46cc6e29535 in_seq=0 out_seq=0 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.023+0000 7fb3d1ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/208429497 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a776fa41e0 con 0x55a77704ba90 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3d1ffb640 10 client.15399 handle_command_reply: tid=0 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 client.15399 shutdown 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 msgr2=0x55a777038fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 0x55a777038fe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0ca1c0 tx=0x7fb3dc03b6b0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 shutdown_connections 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a77704de80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a77703c0f0 0x55a777039a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3d1ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:56.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777031830 0x55a777039520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/208429497 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777030f10 0x55a777038fe0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 >> 172.21.15.3:0/208429497 conn(0x55a77702d420 msgr2=0x55a777033150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 shutdown_connections 2024-05-08T05:54:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 1 -- 172.21.15.3:0/208429497 wait complete. 2024-05-08T05:54:56.032 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.024+0000 7fb3f37b5740 20 client.15399 trim_cache size 0 max 16384 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.055+0000 7fb3f37b5740 1 Processor -- start 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.055+0000 7fb3f37b5740 1 -- start start 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a777039880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77702c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a777053720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1aff30 con 0x55a777031830 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1b27d0 con 0x55a77703c0f0 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1b05c0 con 0x55a777030f10 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77702c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77702c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60656/0 (socket says 172.21.15.3:60656) 2024-05-08T05:54:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 -- 172.21.15.3:0/3130187427 learned_addr learned my addr 172.21.15.3:0/3130187427 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a777053720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a777039880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 -- 172.21.15.3:0/3130187427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 msgr2=0x55a777039880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a777039880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 -- 172.21.15.3:0/3130187427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 msgr2=0x55a777053720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a777053720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 -- 172.21.15.3:0/3130187427 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a77704e1f0 con 0x55a777031830 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77702c400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55a77702e840 tx=0x7fb3e0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e91ceac72e96af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.056+0000 7fb3d3fff640 1 -- 172.21.15.3:0/3130187427 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0013e30 con 0x55a777031830 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3d3fff640 1 -- 172.21.15.3:0/3130187427 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e001d8e0 con 0x55a777031830 2024-05-08T05:54:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3d3fff640 1 -- 172.21.15.3:0/3130187427 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0011af0 con 0x55a777031830 2024-05-08T05:54:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3130187427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 msgr2=0x55a77702c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77702c400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55a77702e840 tx=0x7fb3e0013830 comp rx=0 tx=0).stop 2024-05-08T05:54:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3130187427 shutdown_connections 2024-05-08T05:54:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a77703c0f0 0x55a777053720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a777031830 0x55a77702c400 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.062 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/3130187427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777030f10 0x55a777039880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3130187427 >> 172.21.15.3:0/3130187427 conn(0x55a77702d420 msgr2=0x55a77702bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3130187427 shutdown_connections 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.057+0000 7fb3f37b5740 1 -- 172.21.15.3:0/3130187427 wait complete. 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 -- start start 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 Processor -- start 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777030f10 0x55a777036170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777031830 0x55a7770366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 0x55a777036bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1b0c90 con 0x55a77703c0f0 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1b2100 con 0x55a777030f10 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ec1b1360 con 0x55a777031830 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3f37b5740 10 client.0 _ll_register_callbacks cb 0x7fb3e000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3ead9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 0x55a777036bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.063 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3ead9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 0x55a777036bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60662/0 (socket says 172.21.15.3:60662) 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3ead9d640 1 -- 172.21.15.3:0/2094005338 learned_addr learned my addr 172.21.15.3:0/2094005338 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777031830 0x55a7770366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.058+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777030f10 0x55a777036170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ead9d640 1 -- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777031830 msgr2=0x55a7770366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777031830 0x55a7770366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ead9d640 1 -- 172.21.15.3:0/2094005338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777030f10 msgr2=0x55a777036170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777030f10 0x55a777036170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ead9d640 1 -- 172.21.15.3:0/2094005338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7770b54d0 con 0x55a77703c0f0 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777030f10 0x55a777036170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3e9d9b640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777031830 0x55a7770366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3ead9d640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 0x55a777036bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4010920 tx=0x7fb3e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ce63b298b0e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:54:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/2094005338 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e402e060 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/2094005338 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb3e4003c40 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e001d8e0 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/2094005338 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb3e4035050 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3f37b5740 10 client.15405 fetch_fsmap learned FSMap version 23 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a7770b6830 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/2094005338 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e001d8e0 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.059+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/2094005338 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7fb3e401e670 con 0x55a77703c0f0 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3f37b5740 10 client.15405 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3f37b5740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3f37b5740 20 client.15405 populate_metadata read hostname 'smithi003' 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a7770b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3f37b5740 4 client.15405 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:54:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a776f9e050 con 0x55a77704ba90 2024-05-08T05:54:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a7770b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3d1ffb640 10 client.15405 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.060+0000 7fb3ea59c640 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a7770b9050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16cd7d963eaaaa0c server_cookie=76edc86c0e4cf574 in_seq=0 out_seq=0 2024-05-08T05:54:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3d1ffb640 1 -- 172.21.15.3:0/2094005338 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a776f9e050 con 0x55a77704ba90 2024-05-08T05:54:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3d1ffb640 10 client.15405 handle_command_reply: tid=0 2024-05-08T05:54:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 client.15405 shutdown 2024-05-08T05:54:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 msgr2=0x55a777036bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 0x55a777036bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3e4010920 tx=0x7fb3e4004a40 comp rx=0 tx=0).stop 2024-05-08T05:54:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 shutdown_connections 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55a77704ba90 0x55a7770b9050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55a77703c0f0 0x55a777036bf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a777031830 0x55a7770366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3d1ffb640 0 client.15405 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 --2- 172.21.15.3:0/2094005338 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a777030f10 0x55a777036170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 >> 172.21.15.3:0/2094005338 conn(0x55a77702d420 msgr2=0x55a77702bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 shutdown_connections 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 1 -- 172.21.15.3:0/2094005338 wait complete. 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.064+0000 7fb3f37b5740 20 client.15405 trim_cache size 0 max 16384 2024-05-08T05:54:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fb3d40b2a70 msgr2=0x7fb3d40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fb3d40b2a70 0x7fb3d40b4f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec06b2b0 tx=0x7fb3e40096a0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 msgr2=0x7fb3ec101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec101870 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc005940 tx=0x7fb3dc0042f0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 shutdown_connections 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fb3d40b2a70 0x7fb3d40b4f30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb3ec10cff0 0x7fb3ec1022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fb3ec104540 0x7fb3ec101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 --2- 172.21.15.3:0/2085920455 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fb3ec103c20 0x7fb3ec101870 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 >> 172.21.15.3:0/2085920455 conn(0x7fb3ec0789d0 msgr2=0x7fb3ec0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 shutdown_connections 2024-05-08T05:54:56.070 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:54:56.065+0000 7fb3f09c1640 1 -- 172.21.15.3:0/2085920455 wait complete. 2024-05-08T05:54:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:55 smithi003 ceph-mon[30137]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:57.183 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:54:57.438 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:54:57.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:57 smithi003 ceph-mon[30137]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:57 smithi089 ceph-mon[34495]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:58.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:57 smithi104 ceph-mon[34559]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-08T05:54:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 -- 172.21.15.3:0/1434920183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5954109510 msgr2=0x7f59541098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 --2- 172.21.15.3:0/1434920183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5954109510 0x7f59541098f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5948009990 tx=0x7f5948009ca0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 -- 172.21.15.3:0/1434920183 shutdown_connections 2024-05-08T05:54:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 --2- 172.21.15.3:0/1434920183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f595406bb40 0x7f5954110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 --2- 172.21.15.3:0/1434920183 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5954109510 0x7f59541098f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.316 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 --2- 172.21.15.3:0/1434920183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406b180 0x7f595406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 -- 172.21.15.3:0/1434920183 >> 172.21.15.3:0/1434920183 conn(0x7f5954079350 msgr2=0x7f5954079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 -- 172.21.15.3:0/1434920183 shutdown_connections 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.315+0000 7f595ae0f700 1 -- 172.21.15.3:0/1434920183 wait complete. 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 Processor -- start 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 -- start start 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 0x7f595419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406bb40 0x7f595419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.317 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5954109510 0x7f59541998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5954113620 con 0x7f595406b180 2024-05-08T05:54:59.318 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f59541134a0 con 0x7f5954109510 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f595ae0f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59541137a0 con 0x7f595406bb40 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 0x7f595419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 0x7f595419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60676/0 (socket says 172.21.15.3:60676) 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 -- 172.21.15.3:0/3053387675 learned_addr learned my addr 172.21.15.3:0/3053387675 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5953fff700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406bb40 0x7f595419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f59593ac700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5954109510 0x7f59541998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 -- 172.21.15.3:0/3053387675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406bb40 msgr2=0x7f595419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406bb40 0x7f595419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 -- 172.21.15.3:0/3053387675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5954109510 msgr2=0x7f59541998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5954109510 0x7f59541998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.316+0000 7f5958bab700 1 -- 172.21.15.3:0/3053387675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595419a040 con 0x7f595406b180 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f5953fff700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406bb40 0x7f595419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f5958bab700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 0x7f595419f720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5948000c00 tx=0x7f59480047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:59.319 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948027070 con 0x7f595406b180 2024-05-08T05:54:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5948003f20 con 0x7f595406b180 2024-05-08T05:54:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5948022ca0 con 0x7f595406b180 2024-05-08T05:54:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595419a2d0 con 0x7f595406b180 2024-05-08T05:54:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.317+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59541023a0 con 0x7f595406b180 2024-05-08T05:54:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.318+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595410b9f0 con 0x7f595406b180 2024-05-08T05:54:59.320 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.321+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f594802a070 con 0x7f595406b180 2024-05-08T05:54:59.321 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.321+0000 7f5951ffb700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f59440b2b10 0x7f59440b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:54:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.322+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f59480e4180 con 0x7f595406b180 2024-05-08T05:54:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.322+0000 7f5953fff700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f59440b2b10 0x7f59440b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:54:59.322 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.322+0000 7f5953fff700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f59440b2b10 0x7f59440b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5954079950 tx=0x7f594000b410 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:54:59.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.325+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5948031070 con 0x7f595406b180 2024-05-08T05:54:59.602 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.602+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f595419ae10 con 0x7f595406b180 2024-05-08T05:54:59.603 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.603+0000 7f5951ffb700 1 -- 172.21.15.3:0/3053387675 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7f5948013070 con 0x7f595406b180 2024-05-08T05:54:59.603 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:54:59.603 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:54:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f59440b2b10 msgr2=0x7f59440b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f59440b2b10 0x7f59440b4fd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5954079950 tx=0x7f594000b410 comp rx=0 tx=0).stop 2024-05-08T05:54:59.606 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 msgr2=0x7f595419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 0x7f595419f720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5948000c00 tx=0x7f59480047b0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 shutdown_connections 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5954109510 0x7f59541998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f595406b180 0x7f595419f720 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f59440b2b10 0x7f59440b4fd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 --2- 172.21.15.3:0/3053387675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f595406bb40 0x7f595419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 >> 172.21.15.3:0/3053387675 conn(0x7f5954079350 msgr2=0x7f5954100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 shutdown_connections 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:54:59.606+0000 7f595ae0f700 1 -- 172.21.15.3:0/3053387675 wait complete. 2024-05-08T05:54:59.607 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T05:54:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:59 smithi003 ceph-mon[30137]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-05-08T05:54:59.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:54:59 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3053387675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:59 smithi089 ceph-mon[34495]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-05-08T05:55:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:54:59 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3053387675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:00.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:59 smithi104 ceph-mon[34559]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-05-08T05:55:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:54:59 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3053387675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:00.421 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24493 quiesce db --set-id b04c9cab --release --await' 2024-05-08T05:55:00.421 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24493 quiesce db --set-id b04c9cab --release --await 2024-05-08T05:55:00.689 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:01.817 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:01 smithi003 ceph-mon[30137]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:02.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:01 smithi089 ceph-mon[34495]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:02.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:01 smithi104 ceph-mon[34559]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:02.378 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 -- 172.21.15.3:0/3456533752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810108580 msgr2=0x7f7810108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 --2- 172.21.15.3:0/3456533752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810108580 0x7f7810108960 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7804009990 tx=0x7f7804009ca0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 -- 172.21.15.3:0/3456533752 shutdown_connections 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 --2- 172.21.15.3:0/3456533752 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7810102820 0x7f7810110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 --2- 172.21.15.3:0/3456533752 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810108580 0x7f7810108960 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 --2- 172.21.15.3:0/3456533752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7810101e60 0x7f78101022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 -- 172.21.15.3:0/3456533752 >> 172.21.15.3:0/3456533752 conn(0x7f78100776c0 msgr2=0x7f7810077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 -- 172.21.15.3:0/3456533752 shutdown_connections 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.376+0000 7f7816bc1700 1 -- 172.21.15.3:0/3456533752 wait complete. 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 Processor -- start 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 -- start start 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7810101e60 0x7f781019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 0x7f781019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.379 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7810108580 0x7f7810199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7810113670 con 0x7f7810102820 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f78101134f0 con 0x7f7810101e60 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f7816bc1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78101137f0 con 0x7f7810108580 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f780ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 0x7f781019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f780ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 0x7f781019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60702/0 (socket says 172.21.15.3:60702) 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f780ffff700 1 -- 172.21.15.3:0/2295079828 learned_addr learned my addr 172.21.15.3:0/2295079828 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.377+0000 7f781495d700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7810101e60 0x7f781019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f781515e700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7810108580 0x7f7810199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780ffff700 1 -- 172.21.15.3:0/2295079828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7810108580 msgr2=0x7f7810199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780ffff700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7810108580 0x7f7810199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780ffff700 1 -- 172.21.15.3:0/2295079828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7810101e60 msgr2=0x7f781019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780ffff700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7810101e60 0x7f781019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780ffff700 1 -- 172.21.15.3:0/2295079828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781019a090 con 0x7f7810102820 2024-05-08T05:55:02.380 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780ffff700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 0x7f781019fcb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00d950 tx=0x7f77fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780dffb700 1 -- 172.21.15.3:0/2295079828 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc016070 con 0x7f7810102820 2024-05-08T05:55:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781019a380 con 0x7f7810102820 2024-05-08T05:55:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780dffb700 1 -- 172.21.15.3:0/2295079828 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc00fd40 con 0x7f7810102820 2024-05-08T05:55:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.378+0000 7f780dffb700 1 -- 172.21.15.3:0/2295079828 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc00e3b0 con 0x7f7810102820 2024-05-08T05:55:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.379+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101a6640 con 0x7f7810102820 2024-05-08T05:55:02.381 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.381+0000 7f780dffb700 1 -- 172.21.15.3:0/2295079828 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f77fc019070 con 0x7f7810102820 2024-05-08T05:55:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.382+0000 7f780dffb700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78000b2b10 0x7f78000b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.382+0000 7f781495d700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78000b2b10 0x7f78000b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.382+0000 7f780dffb700 1 -- 172.21.15.3:0/2295079828 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f77fc0e2770 con 0x7f7810102820 2024-05-08T05:55:02.382 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.382+0000 7f7821454b80 1 Processor -- start 2024-05-08T05:55:02.383 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.382+0000 7f7821454b80 1 -- start start 2024-05-08T05:55:02.387 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f7821454b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77ecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f7821454b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ed460 0x5578f77e6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f7821454b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 0x5578f77e7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781495d700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78000b2b10 0x7f78000b4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f780400b5c0 tx=0x7f7804005960 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f7821454b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5578f7734a40 con 0x5578f77e73c0 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f7821454b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5578f7699ea0 con 0x5578f77ecb40 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f7821454b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578f7730a00 con 0x5578f77ed460 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 0x5578f77e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 0x5578f77e7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60718/0 (socket says 172.21.15.3:60718) 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 -- 172.21.15.3:0/2875410206 learned_addr learned my addr 172.21.15.3:0/2875410206 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781495d700 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77ecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f780ffff700 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ed460 0x5578f77e6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 -- 172.21.15.3:0/2875410206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ed460 msgr2=0x5578f77e6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.388 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ed460 0x5578f77e6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 -- 172.21.15.3:0/2875410206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 msgr2=0x5578f77ecf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77ecf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 -- 172.21.15.3:0/2875410206 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f76819d0 con 0x5578f77e73c0 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781495d700 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.383+0000 7f781515e700 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 0x5578f77e7800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f78080098f0 tx=0x7f7808009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0fda8f2d6caa684 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.384+0000 7f77faffd700 1 -- 172.21.15.3:0/2875410206 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808004bb0 con 0x5578f77e73c0 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.384+0000 7f77faffd700 1 -- 172.21.15.3:0/2875410206 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7808004d50 con 0x5578f77e73c0 2024-05-08T05:55:02.389 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.384+0000 7f77faffd700 1 -- 172.21.15.3:0/2875410206 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808007da0 con 0x5578f77e73c0 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.384+0000 7f7821454b80 1 -- 172.21.15.3:0/2875410206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 msgr2=0x5578f77e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.384+0000 7f7821454b80 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 0x5578f77e7800 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f78080098f0 tx=0x7f7808009c00 comp rx=0 tx=0).stop 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 -- 172.21.15.3:0/2875410206 shutdown_connections 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77e73c0 0x5578f77e7800 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77ecf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 --2- 172.21.15.3:0/2875410206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ed460 0x5578f77e6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 -- 172.21.15.3:0/2875410206 >> 172.21.15.3:0/2875410206 conn(0x5578f770ec70 msgr2=0x5578f77dd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 -- 172.21.15.3:0/2875410206 shutdown_connections 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 -- 172.21.15.3:0/2875410206 wait complete. 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 -- start start 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.385+0000 7f7821454b80 1 Processor -- start 2024-05-08T05:55:02.390 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77e73c0 0x5578f77f3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ecb40 0x5578f77f0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5578f77f3e10 con 0x5578f77ed460 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5578f76edfd0 con 0x5578f77e73c0 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578f77f3f90 con 0x5578f77ecb40 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f7821454b80 10 client.0 _ll_register_callbacks cb 0x5578f76c86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781495d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77e73c0 0x5578f77f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781495d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77e73c0 0x5578f77f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:39086/0 (socket says 172.21.15.3:39086) 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781495d700 1 -- 172.21.15.3:0/220491358 learned_addr learned my addr 172.21.15.3:0/220491358 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f780ffff700 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ecb40 0x5578f77f0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.391 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 -- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ecb40 msgr2=0x5578f77f0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ecb40 0x5578f77f0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 -- 172.21.15.3:0/220491358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77e73c0 msgr2=0x5578f77f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77e73c0 0x5578f77f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 -- 172.21.15.3:0/220491358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f76eb860 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.386+0000 7f781515e700 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f0950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7808007a20 tx=0x7f7808004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22bb29a5d0a7ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f77effff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f77effff700 1 -- 172.21.15.3:0/220491358 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808022070 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f77effff700 1 -- 172.21.15.3:0/220491358 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7808007c90 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7808004d50 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f77effff700 1 -- 172.21.15.3:0/220491358 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f780802e050 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 10 client.15423 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5578f77fe670 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f77effff700 1 -- 172.21.15.3:0/220491358 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7808004d50 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f77effff700 1 -- 172.21.15.3:0/220491358 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7808015a60 con 0x5578f77ed460 2024-05-08T05:55:02.392 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 10 client.15423 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 20 client.15423 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f7800ba0 0x5578f7802f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 4 client.15423 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:02.393 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.387+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578f7742420 con 0x5578f7800ba0 2024-05-08T05:55:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.388+0000 7f781495d700 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f7800ba0 0x5578f7802f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.388+0000 7f77effff700 10 client.15423 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.388+0000 7f781495d700 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f7800ba0 0x5578f7802f90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a6a556b6c7b34b server_cookie=6313a288afda82d4 in_seq=0 out_seq=0 2024-05-08T05:55:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.389+0000 7f77effff700 1 -- 172.21.15.3:0/220491358 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5578f7742420 con 0x5578f7800ba0 2024-05-08T05:55:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.389+0000 7f77effff700 10 client.15423 handle_command_reply: tid=0 2024-05-08T05:55:02.394 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.389+0000 7f7821454b80 1 client.15423 shutdown 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.389+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 msgr2=0x5578f77f0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.389+0000 7f7821454b80 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f0950 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7808007a20 tx=0x7f7808004ad0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 shutdown_connections 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f0950 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f7800ba0 0x5578f7802f90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77e73c0 0x5578f77f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 --2- 172.21.15.3:0/220491358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77ecb40 0x5578f77f0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f77effff700 0 client.15423 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 >> 172.21.15.3:0/220491358 conn(0x5578f770ec70 msgr2=0x5578f77d7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 shutdown_connections 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 1 -- 172.21.15.3:0/220491358 wait complete. 2024-05-08T05:55:02.395 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.390+0000 7f7821454b80 20 client.15423 trim_cache size 0 max 16384 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.410+0000 7f7821454b80 1 Processor -- start 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.410+0000 7f7821454b80 1 -- start start 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f7821454b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f7821454b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 0x5578f77d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f7821454b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ed460 0x5578f78085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f7821454b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ab1a0 con 0x5578f77ecb40 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f7821454b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ada20 con 0x5578f77ed460 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f7821454b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ab810 con 0x5578f77e73c0 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 0x5578f77d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 0x5578f77d7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60746/0 (socket says 172.21.15.3:60746) 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 -- 172.21.15.3:0/2820876900 learned_addr learned my addr 172.21.15.3:0/2820876900 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f781495d700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.412 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f781515e700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ed460 0x5578f78085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 -- 172.21.15.3:0/2820876900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 msgr2=0x5578f77fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 -- 172.21.15.3:0/2820876900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ed460 msgr2=0x5578f78085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ed460 0x5578f78085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 -- 172.21.15.3:0/2820876900 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578f77e71c0 con 0x5578f77ecb40 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f781515e700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ed460 0x5578f78085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f781495d700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f780ffff700 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 0x5578f77d7ea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0a93e0 tx=0x7f77fc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f71908373fffdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.411+0000 7f77fa7fc700 1 -- 172.21.15.3:0/2820876900 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc014030 con 0x5578f77ecb40 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f77fa7fc700 1 -- 172.21.15.3:0/2820876900 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc0ad800 con 0x5578f77ecb40 2024-05-08T05:55:02.413 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f77fa7fc700 1 -- 172.21.15.3:0/2820876900 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc020040 con 0x5578f77ecb40 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 -- 172.21.15.3:0/2820876900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 msgr2=0x5578f77d7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 0x5578f77d7ea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0a93e0 tx=0x7f77fc010000 comp rx=0 tx=0).stop 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 -- 172.21.15.3:0/2820876900 shutdown_connections 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ed460 0x5578f78085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 --2- 172.21.15.3:0/2820876900 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ecb40 0x5578f77d7ea0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.416 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 -- 172.21.15.3:0/2820876900 >> 172.21.15.3:0/2820876900 conn(0x5578f770ec70 msgr2=0x5578f77e22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 -- 172.21.15.3:0/2820876900 shutdown_connections 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.412+0000 7f7821454b80 1 -- 172.21.15.3:0/2820876900 wait complete. 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.413+0000 7f7821454b80 1 -- start start 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.413+0000 7f7821454b80 1 Processor -- start 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.413+0000 7f7821454b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.413+0000 7f7821454b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77f6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.413+0000 7f7821454b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:60760/0 (socket says 172.21.15.3:60760) 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 -- 172.21.15.3:0/1220904873 learned_addr learned my addr 172.21.15.3:0/1220904873 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781495d700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f78101abee0 con 0x5578f77ed460 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ad350 con 0x5578f77ecb40 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f78101ac5b0 con 0x5578f77e73c0 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f780ffff700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.417 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f7821454b80 10 client.0 _ll_register_callbacks cb 0x5578f76c86b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 -- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 msgr2=0x5578f77f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 -- 172.21.15.3:0/1220904873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 msgr2=0x5578f77f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578f786b340 con 0x5578f77ed460 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f780ffff700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f781515e700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f6c70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7808005bd0 tx=0x7f7808009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936ca2a034a186a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.414+0000 7f77effff700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f77effff700 1 -- 172.21.15.3:0/1220904873 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808037060 con 0x5578f77ed460 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc0ad800 con 0x5578f77ed460 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f77effff700 1 -- 172.21.15.3:0/1220904873 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7808005c60 con 0x5578f77ed460 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f77effff700 1 -- 172.21.15.3:0/1220904873 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f780803e050 con 0x5578f77ed460 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f7821454b80 10 client.15429 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:02.418 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5578f786c620 con 0x5578f77ed460 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f77effff700 1 -- 172.21.15.3:0/1220904873 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc0ad800 con 0x5578f77ed460 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f77effff700 1 -- 172.21.15.3:0/1220904873 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f780801ea50 con 0x5578f77ed460 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f7821454b80 10 client.15429 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.415+0000 7f7821454b80 10 client.15429 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f7821454b80 20 client.15429 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f7821454b80 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f786cc70 0x5578f786f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f7821454b80 4 client.15429 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b04c9cab", "release": true, "await": true}] 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b04c9cab", "release": true, "await": true}) -- 0x5578f773c4c0 con 0x5578f786cc70 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f781495d700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f786cc70 0x5578f786f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f77effff700 10 client.15429 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:02.419 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.416+0000 7f781495d700 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f786cc70 0x5578f786f060 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9134d5e502bc63af server_cookie=49141195e4ce28eb in_seq=0 out_seq=0 2024-05-08T05:55:02.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f77effff700 1 -- 172.21.15.3:0/1220904873 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5578f773c4c0 con 0x5578f786cc70 2024-05-08T05:55:02.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f77effff700 10 client.15429 handle_command_reply: tid=0 2024-05-08T05:55:02.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 client.15429 shutdown 2024-05-08T05:55:02.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 msgr2=0x5578f77f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.420 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f6c70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7808005bd0 tx=0x7f7808009540 comp rx=0 tx=0).stop 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 shutdown_connections 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5578f77ed460 0x5578f77f6c70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578f77e73c0 0x5578f77f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5578f786cc70 0x5578f786f060 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 --2- 172.21.15.3:0/1220904873 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5578f77ecb40 0x5578f77f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 >> 172.21.15.3:0/1220904873 conn(0x5578f770ec70 msgr2=0x5578f77ee9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f77effff700 0 client.15429 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 shutdown_connections 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 1 -- 172.21.15.3:0/1220904873 wait complete. 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.417+0000 7f7821454b80 20 client.15429 trim_cache size 0 max 16384 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 23, 2024-05-08T05:55:02.421 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24493, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 4, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "b04c9cab": { 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 4, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 60.0, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 79.4, 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-05-08T05:55:02.422 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:02.423 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78000b2b10 msgr2=0x7f78000b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78000b2b10 0x7f78000b4fd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f780400b5c0 tx=0x7f7804005960 comp rx=0 tx=0).stop 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 msgr2=0x7f781019fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 0x7f781019fcb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00d950 tx=0x7f77fc00dc60 comp rx=0 tx=0).stop 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 shutdown_connections 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f78000b2b10 0x7f78000b4fd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7810108580 0x7f7810199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7810101e60 0x7f781019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 --2- 172.21.15.3:0/2295079828 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7810102820 0x7f781019fcb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 >> 172.21.15.3:0/2295079828 conn(0x7f78100776c0 msgr2=0x7f78101009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:02.424 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 shutdown_connections 2024-05-08T05:55:02.425 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:02.418+0000 7f7816bc1700 1 -- 172.21.15.3:0/2295079828 wait complete. 2024-05-08T05:55:03.246 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b04c9cab', seconds elapsed: 31.4 2024-05-08T05:55:03.246 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-05-08T05:55:03.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:55:03.506 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:03.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:03 smithi003 ceph-mon[30137]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:55:04.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:03 smithi089 ceph-mon[34495]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:55:04.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:03 smithi104 ceph-mon[34559]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:55:05.274 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 -- 172.21.15.3:0/4027305892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b006b180 msgr2=0x7ff7b006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/4027305892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b006b180 0x7ff7b006b600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff798009990 tx=0x7ff798009ca0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 -- 172.21.15.3:0/4027305892 shutdown_connections 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/4027305892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7b006bb40 0x7ff7b0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/4027305892 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b0109510 0x7ff7b01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/4027305892 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b006b180 0x7ff7b006b600 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 -- 172.21.15.3:0/4027305892 >> 172.21.15.3:0/4027305892 conn(0x7ff7b0079350 msgr2=0x7ff7b0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 -- 172.21.15.3:0/4027305892 shutdown_connections 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.274+0000 7ff7b51e6700 1 -- 172.21.15.3:0/4027305892 wait complete. 2024-05-08T05:55:05.275 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 Processor -- start 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 -- start start 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b006b180 0x7ff7b019bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7b006bb40 0x7ff7b019c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 0x7ff7b01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0113620 con 0x7ff7b0109510 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01134a0 con 0x7ff7b006b180 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7b51e6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01137a0 con 0x7ff7b006bb40 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7af59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 0x7ff7b01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7af59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 0x7ff7b01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47952/0 (socket says 172.21.15.3:47952) 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7aed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b006b180 0x7ff7b019bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.275+0000 7ff7af59e700 1 -- 172.21.15.3:0/2175251767 learned_addr learned my addr 172.21.15.3:0/2175251767 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:05.277 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7a659c700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7b006bb40 0x7ff7b019c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7aed9d700 1 -- 172.21.15.3:0/2175251767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7b006bb40 msgr2=0x7ff7b019c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7aed9d700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7b006bb40 0x7ff7b019c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7aed9d700 1 -- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 msgr2=0x7ff7b01a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7aed9d700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 0x7ff7b01a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7aed9d700 1 -- 172.21.15.3:0/2175251767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b01a0d80 con 0x7ff7b006b180 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7af59e700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 0x7ff7b01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7aed9d700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b006b180 0x7ff7b019bd70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e9f0 tx=0x7ff7a000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000c9c0 con 0x7ff7b006b180 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7a000cb60 con 0x7ff7b006b180 2024-05-08T05:55:05.278 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.276+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b01a1070 con 0x7ff7b006b180 2024-05-08T05:55:05.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.277+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0018680 con 0x7ff7b006b180 2024-05-08T05:55:05.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.277+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b01a88f0 con 0x7ff7b006b180 2024-05-08T05:55:05.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.278+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b006a190 con 0x7ff7b006b180 2024-05-08T05:55:05.279 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.279+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7ff7a0010430 con 0x7ff7b006b180 2024-05-08T05:55:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.280+0000 7ff7acd99700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff79c0b2b90 0x7ff79c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.281+0000 7ff7a659c700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff79c0b2b90 0x7ff79c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.281+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff7a00e2c80 con 0x7ff7b006b180 2024-05-08T05:55:05.281 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.281+0000 7ff7a659c700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff79c0b2b90 0x7ff79c0b5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff79800b2e0 tx=0x7ff798005960 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:05.285 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.285+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7a00e9080 con 0x7ff7b006b180 2024-05-08T05:55:05.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.558+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7b01a8b20 con 0x7ff7b006b180 2024-05-08T05:55:05.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.559+0000 7ff7acd99700 1 -- 172.21.15.3:0/2175251767 <== mon.2 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7ff7a00aa0a0 con 0x7ff7b006b180 2024-05-08T05:55:05.560 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:55:05.560 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:55:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff79c0b2b90 msgr2=0x7ff79c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.562 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff79c0b2b90 0x7ff79c0b5050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff79800b2e0 tx=0x7ff798005960 comp rx=0 tx=0).stop 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b006b180 msgr2=0x7ff7b019bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b006b180 0x7ff7b019bd70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e9f0 tx=0x7ff7a000ed00 comp rx=0 tx=0).stop 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 shutdown_connections 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff7b0109510 0x7ff7b01a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7b006b180 0x7ff7b019bd70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff79c0b2b90 0x7ff79c0b5050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 --2- 172.21.15.3:0/2175251767 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7b006bb40 0x7ff7b019c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 >> 172.21.15.3:0/2175251767 conn(0x7ff7b0079350 msgr2=0x7ff7b010e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 shutdown_connections 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:05.562+0000 7ff7b51e6700 1 -- 172.21.15.3:0/2175251767 wait complete. 2024-05-08T05:55:05.563 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T05:55:05.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:05 smithi003 ceph-mon[30137]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:05.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:05 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2175251767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:06.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:05 smithi089 ceph-mon[34495]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:06.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:05 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2175251767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:06.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:05 smithi104 ceph-mon[34559]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:06.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:05 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2175251767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:06.389 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24493 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0' 2024-05-08T05:55:06.389 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24493 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0 2024-05-08T05:55:06.679 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:07.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:07 smithi003 ceph-mon[30137]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:08.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:07 smithi089 ceph-mon[34495]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:08.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:07 smithi104 ceph-mon[34559]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 -- 172.21.15.3:0/3412685801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 msgr2=0x7ff1cc110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/3412685801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc110390 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0099c0 tx=0x7ff1bc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 -- 172.21.15.3:0/3412685801 shutdown_connections 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/3412685801 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc110390 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/3412685801 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc108490 0x7ff1cc108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/3412685801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.426+0000 7ff1d15b8700 1 -- 172.21.15.3:0/3412685801 >> 172.21.15.3:0/3412685801 conn(0x7ff1cc07a650 msgr2=0x7ff1cc07aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.427 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.427+0000 7ff1d15b8700 1 -- 172.21.15.3:0/3412685801 shutdown_connections 2024-05-08T05:55:08.428 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.427+0000 7ff1d15b8700 1 -- 172.21.15.3:0/3412685801 wait complete. 2024-05-08T05:55:08.429 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.427+0000 7ff1d15b8700 1 Processor -- start 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 -- start start 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc108490 0x7ff1cc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc1136f0 con 0x7ff1cc103eb0 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc113570 con 0x7ff1cc1034f0 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1d15b8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc113870 con 0x7ff1cc108490 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:42500/0 (socket says 172.21.15.3:42500) 2024-05-08T05:55:08.430 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47974/0 (socket says 172.21.15.3:47974) 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1caffd700 1 -- 172.21.15.3:0/1342467672 learned_addr learned my addr 172.21.15.3:0/1342467672 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.428+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc108490 0x7ff1cc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1caffd700 1 -- 172.21.15.3:0/1342467672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc108490 msgr2=0x7ff1cc076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc108490 0x7ff1cc076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1caffd700 1 -- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 msgr2=0x7ff1cc075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1caffd700 1 -- 172.21.15.3:0/1342467672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc1a7fd0 con 0x7ff1cc1034f0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc0757a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400efc0 tx=0x7ff1c400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1b3fff700 1 -- 172.21.15.3:0/1342467672 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40092e0 con 0x7ff1cc1034f0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc1997e0 con 0x7ff1cc1034f0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc199d70 con 0x7ff1cc1034f0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.429+0000 7ff1b3fff700 1 -- 172.21.15.3:0/1342467672 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c400c9c0 con 0x7ff1cc1034f0 2024-05-08T05:55:08.431 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.430+0000 7ff1b3fff700 1 -- 172.21.15.3:0/1342467672 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004580 con 0x7ff1cc1034f0 2024-05-08T05:55:08.432 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.432+0000 7ff1b3fff700 1 -- 172.21.15.3:0/1342467672 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7ff1c401b030 con 0x7ff1cc1034f0 2024-05-08T05:55:08.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.433+0000 7ff1b3fff700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff1b40b2c40 0x7ff1b40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.433 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.434+0000 7ff1b3fff700 1 -- 172.21.15.3:0/1342467672 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff1c40e28d0 con 0x7ff1cc1034f0 2024-05-08T05:55:08.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.434+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff1b40b2c40 0x7ff1b40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.434 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.434+0000 7ff1dbe4bb80 1 Processor -- start 2024-05-08T05:55:08.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.434+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff1b40b2c40 0x7ff1b40b5100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc07ac30 tx=0x7ff1c0018040 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.435+0000 7ff1dbe4bb80 1 -- start start 2024-05-08T05:55:08.435 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.435+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef197410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.435+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47984/0 (socket says 172.21.15.3:47984) 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1caffd700 1 -- 172.21.15.3:0/1323963597 learned_addr learned my addr 172.21.15.3:0/1323963597 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef197950 0x5559ef192830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef192e50 0x5559ef1932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5559ef0e49f0 con 0x5559ef197030 2024-05-08T05:55:08.437 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5559ef083950 con 0x5559ef192e50 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef197950 0x5559ef192830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559ef0e09b0 con 0x5559ef197950 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1ca7fc700 1 -- 172.21.15.3:0/1323963597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef192e50 msgr2=0x5559ef1932b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.436+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef192e50 0x5559ef1932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1ca7fc700 1 -- 172.21.15.3:0/1323963597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 msgr2=0x5559ef197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1ca7fc700 1 -- 172.21.15.3:0/1323963597 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ef031a60 con 0x5559ef197950 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef197950 0x5559ef192830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009f50 tx=0x7ff1c001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4ec4fdad92f3f24 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.438 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1b17fa700 1 -- 172.21.15.3:0/1323963597 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0015e30 con 0x5559ef197950 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1b17fa700 1 -- 172.21.15.3:0/1323963597 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c0004830 con 0x5559ef197950 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.437+0000 7ff1b17fa700 1 -- 172.21.15.3:0/1323963597 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c00054c0 con 0x5559ef197950 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef197950 msgr2=0x5559ef192830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef197950 0x5559ef192830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009f50 tx=0x7ff1c001c000 comp rx=0 tx=0).stop 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 shutdown_connections 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef192e50 0x5559ef1932b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1323963597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef197950 0x5559ef192830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 >> 172.21.15.3:0/1323963597 conn(0x5559ef0be290 msgr2=0x5559ef18d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 shutdown_connections 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1323963597 wait complete. 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 -- start start 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.438+0000 7ff1dbe4bb80 1 Processor -- start 2024-05-08T05:55:08.440 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef19e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5559ef19f4c0 con 0x5559ef197950 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5559ef04adc0 con 0x5559ef197030 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559ef19f640 con 0x5559ef192e50 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1dbe4bb80 10 client.0 _ll_register_callbacks cb 0x5559ef07cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47986/0 (socket says 172.21.15.3:47986) 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1615011205 learned_addr learned my addr 172.21.15.3:0/1615011205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 msgr2=0x5559ef19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1615011205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 msgr2=0x5559ef19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1615011205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ef080820 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.439+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef19bf50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc003680 tx=0x7ff1bc005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d66d756201259a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1615011205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc009e60 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1615011205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff1bc003bd0 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c0004830 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1615011205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc009590 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1dbe4bb80 10 client.15447 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.440+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5559ef1bacb0 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1615011205 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c0004830 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1615011205 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7ff1bc00f660 con 0x5559ef197950 2024-05-08T05:55:08.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1dbe4bb80 10 client.15447 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1dbe4bb80 10 client.15447 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1dbe4bb80 20 client.15447 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef1bd420 0x5559ef1bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1dbe4bb80 4 client.15447 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559ef15e7e0 con 0x5559ef1bd420 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.441+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef1bd420 0x5559ef1bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.442+0000 7ff1aeff5700 10 client.15447 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.442+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef1bd420 0x5559ef1bf880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c521f4375bcc1798 server_cookie=78b4f466fd611ed5 in_seq=0 out_seq=0 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.442+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1615011205 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559ef15e7e0 con 0x5559ef1bd420 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.442+0000 7ff1aeff5700 10 client.15447 handle_command_reply: tid=0 2024-05-08T05:55:08.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.442+0000 7ff1dbe4bb80 1 client.15447 shutdown 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 msgr2=0x5559ef19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef19bf50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc003680 tx=0x7ff1bc005be0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 shutdown_connections 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef19bf50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef1bd420 0x5559ef1bf880 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1615011205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1aeff5700 0 client.15447 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 >> 172.21.15.3:0/1615011205 conn(0x5559ef0be290 msgr2=0x5559ef198680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 shutdown_connections 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1615011205 wait complete. 2024-05-08T05:55:08.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.443+0000 7ff1dbe4bb80 20 client.15447 trim_cache size 0 max 16384 2024-05-08T05:55:08.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 Processor -- start 2024-05-08T05:55:08.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 -- start start 2024-05-08T05:55:08.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef1c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197950 0x5559ef195c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.468 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800cc00 con 0x5559ef197030 2024-05-08T05:55:08.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800f4a0 con 0x5559ef197950 2024-05-08T05:55:08.469 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800d290 con 0x5559ef192e50 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef1c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1cb7fe700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197950 0x5559ef195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1ca7fc700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef1c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47990/0 (socket says 172.21.15.3:47990) 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.468+0000 7ff1ca7fc700 1 -- 172.21.15.3:0/1317216223 learned_addr learned my addr 172.21.15.3:0/1317216223 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1317216223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 msgr2=0x5559ef1c5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1c5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1317216223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 msgr2=0x5559ef1c56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef1c56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1cb7fe700 1 -- 172.21.15.3:0/1317216223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ef196c60 con 0x5559ef197950 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef1c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:08.470 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197950 0x5559ef195c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009750 tx=0x7ff1bc0057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f236d210115ab336 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.469+0000 7ff1b0ff9700 1 -- 172.21.15.3:0/1317216223 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00f660 con 0x5559ef197950 2024-05-08T05:55:08.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.470+0000 7ff1b0ff9700 1 -- 172.21.15.3:0/1317216223 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1bc038070 con 0x5559ef197950 2024-05-08T05:55:08.471 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.470+0000 7ff1b0ff9700 1 -- 172.21.15.3:0/1317216223 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc00be40 con 0x5559ef197950 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.470+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1317216223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197950 msgr2=0x5559ef195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.470+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197950 0x5559ef195c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009750 tx=0x7ff1bc0057a0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1317216223 shutdown_connections 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197950 0x5559ef195c70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1c5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1317216223 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197030 0x5559ef1c56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1317216223 >> 172.21.15.3:0/1317216223 conn(0x5559ef0be290 msgr2=0x5559ef19b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1317216223 shutdown_connections 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1317216223 wait complete. 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.471+0000 7ff1dbe4bb80 1 -- start start 2024-05-08T05:55:08.472 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 Processor -- start 2024-05-08T05:55:08.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef1b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef197d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800d960 con 0x5559ef197950 2024-05-08T05:55:08.473 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800edd0 con 0x5559ef197030 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b800e030 con 0x5559ef192e50 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1dbe4bb80 10 client.0 _ll_register_callbacks cb 0x5559ef07cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1caffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:54558/0 (socket says 172.21.15.3:54558) 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1caffd700 1 -- 172.21.15.3:0/1771318419 learned_addr learned my addr 172.21.15.3:0/1771318419 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef1b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.472+0000 7ff1cb7fe700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1caffd700 1 -- 172.21.15.3:0/1771318419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 msgr2=0x5559ef1b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef1b5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1caffd700 1 -- 172.21.15.3:0/1771318419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 msgr2=0x5559ef197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1caffd700 1 -- 172.21.15.3:0/1771318419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ef1985a0 con 0x5559ef192e50 2024-05-08T05:55:08.474 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1caffd700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1b59f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40a9800 tx=0x7ff1c4004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c5fdbe6a169715 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1771318419 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40c3060 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1771318419 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff1c40b2190 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.473+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1bc038070 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1771318419 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff1c40b2410 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 10 client.25207 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5559ef1998c0 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1771318419 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c40e2440 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1771318419 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7ff1c40adbd0 con 0x5559ef192e50 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 10 client.25207 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:08.476 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 10 client.25207 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 20 client.25207 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef229070 0x5559ef22b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 4 client.25207 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}] 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}) -- 0x7ff1bc038070 con 0x5559ef229070 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.474+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef229070 0x5559ef22b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.475+0000 7ff1ca7fc700 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef229070 0x5559ef22b490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee2bf0eb551b1bc server_cookie=88397c6e4b4c9912 in_seq=0 out_seq=0 2024-05-08T05:55:08.477 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.475+0000 7ff1aeff5700 10 client.25207 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:08.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1aeff5700 1 -- 172.21.15.3:0/1771318419 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7ff1bc038070 con 0x5559ef229070 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1aeff5700 10 client.25207 handle_command_reply: tid=0 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 client.25207 shutdown 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 msgr2=0x5559ef1b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1b59f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40a9800 tx=0x7ff1c4004950 comp rx=0 tx=0).stop 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 shutdown_connections 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5559ef197950 0x5559ef197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559ef192e50 0x5559ef1b59f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5559ef229070 0x5559ef22b490 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1dbe4bb80 1 --2- 172.21.15.3:0/1771318419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5559ef197030 0x5559ef1b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.480+0000 7ff1aeff5700 0 client.25207 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:08.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.481+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 >> 172.21.15.3:0/1771318419 conn(0x5559ef0be290 msgr2=0x5559ef195c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.481+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 shutdown_connections 2024-05-08T05:55:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.481+0000 7ff1dbe4bb80 1 -- 172.21.15.3:0/1771318419 wait complete. 2024-05-08T05:55:08.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.481+0000 7ff1dbe4bb80 20 client.25207 trim_cache size 0 max 16384 2024-05-08T05:55:08.482 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:55:08.482 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 23, 2024-05-08T05:55:08.482 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24493, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 6, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "adda3b08": { 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 6, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 60.0, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 79.4, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-05-08T05:55:08.483 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "QUIESCED", 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:55:08.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff1b40b2c40 msgr2=0x7ff1b40b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff1b40b2c40 0x7ff1b40b5100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc07ac30 tx=0x7ff1c0018040 comp rx=0 tx=0).stop 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 msgr2=0x7ff1cc0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc0757a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1c400efc0 tx=0x7ff1c400c3a0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 shutdown_connections 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff1b40b2c40 0x7ff1b40b5100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1cc108490 0x7ff1cc076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff1cc1034f0 0x7ff1cc0757a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 --2- 172.21.15.3:0/1342467672 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff1cc103eb0 0x7ff1cc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 >> 172.21.15.3:0/1342467672 conn(0x7ff1cc07a650 msgr2=0x7ff1cc1015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 shutdown_connections 2024-05-08T05:55:08.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:08.482+0000 7ff1d15b8700 1 -- 172.21.15.3:0/1342467672 wait complete. 2024-05-08T05:55:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 -- 172.21.15.3:0/2095955863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc104540 msgr2=0x7f63cc10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 --2- 172.21.15.3:0/2095955863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc104540 0x7f63cc10ca20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009900 tx=0x7f63c002a480 comp rx=0 tx=0).stop 2024-05-08T05:55:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 -- 172.21.15.3:0/2095955863 shutdown_connections 2024-05-08T05:55:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 --2- 172.21.15.3:0/2095955863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc10cff0 0x7f63cc10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.187 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 --2- 172.21.15.3:0/2095955863 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc104540 0x7f63cc10ca20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 --2- 172.21.15.3:0/2095955863 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63cc103c20 0x7f63cc104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 -- 172.21.15.3:0/2095955863 >> 172.21.15.3:0/2095955863 conn(0x7f63cc0789d0 msgr2=0x7f63cc078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 -- 172.21.15.3:0/2095955863 shutdown_connections 2024-05-08T05:55:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.186+0000 7f63d14cf640 1 -- 172.21.15.3:0/2095955863 wait complete. 2024-05-08T05:55:09.188 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.187+0000 7f63d14cf640 1 Processor -- start 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.187+0000 7f63d14cf640 1 -- start start 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63d14cf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc103c20 0x7f63cc101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63d14cf640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63cc104540 0x7f63cc101db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63d14cf640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 0x7f63cc1022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63d14cf640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc112c30 con 0x7f63cc10cff0 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63d14cf640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc112ab0 con 0x7f63cc104540 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63d14cf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc112db0 con 0x7f63cc103c20 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 0x7f63cc1022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 0x7f63cc1022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48026/0 (socket says 172.21.15.3:48026) 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63cb7fe640 1 -- 172.21.15.3:0/1267465066 learned_addr learned my addr 172.21.15.3:0/1267465066 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63caffd640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc103c20 0x7f63cc101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63cc104540 0x7f63cc101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63caffd640 1 -- 172.21.15.3:0/1267465066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63cc104540 msgr2=0x7f63cc101db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63caffd640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63cc104540 0x7f63cc101db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63caffd640 1 -- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 msgr2=0x7f63cc1022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63caffd640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 0x7f63cc1022f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63caffd640 1 -- 172.21.15.3:0/1267465066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc1a76b0 con 0x7f63cc103c20 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.188+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 0x7f63cc1022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.189+0000 7f63caffd640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc103c20 0x7f63cc101870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00d6d0 tx=0x7f63bc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.189+0000 7f63b3fff640 1 -- 172.21.15.3:0/1267465066 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc017070 con 0x7f63cc103c20 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.189+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc1a79a0 con 0x7f63cc103c20 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.189+0000 7f63b3fff640 1 -- 172.21.15.3:0/1267465066 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc009820 con 0x7f63cc103c20 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.189+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc1a7f10 con 0x7f63cc103c20 2024-05-08T05:55:09.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.189+0000 7f63b3fff640 1 -- 172.21.15.3:0/1267465066 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc004c40 con 0x7f63cc103c20 2024-05-08T05:55:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.192+0000 7f63d41e2740 1 Processor -- start 2024-05-08T05:55:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.192+0000 7f63b3fff640 1 -- 172.21.15.3:0/1267465066 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 25) ==== 149884+0+0 (secure 0 0 0) 0x7f63bc005220 con 0x7f63cc103c20 2024-05-08T05:55:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.192+0000 7f63d41e2740 1 -- start start 2024-05-08T05:55:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a75a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a75a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63d41e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5584a74fb5d0 con 0x5584a75a10f0 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63d41e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5584a74b4890 con 0x5584a7596830 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63d41e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5584a74f7590 con 0x5584a7595f10 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63ca7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a75a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:54586/0 (socket says 172.21.15.3:54586) 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63cb7fe640 1 -- 172.21.15.3:0/3626781778 learned_addr learned my addr 172.21.15.3:0/3626781778 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63caffd640 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a75a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63b3fff640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f63b40b2af0 0x7f63b40b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.193+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f63b40b2af0 0x7f63b40b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.196 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63caffd640 1 -- 172.21.15.3:0/3626781778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 msgr2=0x5584a75962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63b3fff640 1 -- 172.21.15.3:0/1267465066 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f63bc0e2610 con 0x7f63cc103c20 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63caffd640 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63caffd640 1 -- 172.21.15.3:0/3626781778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 msgr2=0x5584a75a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63caffd640 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a75a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63caffd640 1 -- 172.21.15.3:0/3626781778 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a720d210 con 0x5584a7596830 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a75a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63caffd640 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a75a0bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0df870 tx=0x7f63bc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d6d222f3a411ed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f63b40b2af0 0x7f63b40b4fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005c40 tx=0x7f63c4004ab0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63b17fa640 1 -- 172.21.15.3:0/3626781778 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc020070 con 0x5584a7596830 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63b17fa640 1 -- 172.21.15.3:0/3626781778 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc0a93e0 con 0x5584a7596830 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.194+0000 7f63b17fa640 1 -- 172.21.15.3:0/3626781778 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc009af0 con 0x5584a7596830 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 -- 172.21.15.3:0/3626781778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 msgr2=0x5584a75a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.197 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a75a0bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0df870 tx=0x7f63bc005850 comp rx=0 tx=0).stop 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 -- 172.21.15.3:0/3626781778 shutdown_connections 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a75a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a75a0bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3626781778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 -- 172.21.15.3:0/3626781778 >> 172.21.15.3:0/3626781778 conn(0x5584a7592420 msgr2=0x5584a7592800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 -- 172.21.15.3:0/3626781778 shutdown_connections 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 -- 172.21.15.3:0/3626781778 wait complete. 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.195+0000 7f63d41e2740 1 -- start start 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 Processor -- start 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7595f10 0x5584a759e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7596830 0x5584a759e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.198 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a759ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.200 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5584a75a3cc0 con 0x5584a75a10f0 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5584a74b8370 con 0x5584a7595f10 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5584a75a3e40 con 0x5584a7596830 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a759ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63d41e2740 10 client.0 _ll_register_callbacks cb 0x5584a720ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63cb7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a759ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48050/0 (socket says 172.21.15.3:48050) 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63cb7fe640 1 -- 172.21.15.3:0/3500289907 learned_addr learned my addr 172.21.15.3:0/3500289907 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63caffd640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7595f10 0x5584a759e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7596830 0x5584a759e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63caffd640 1 -- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7596830 msgr2=0x5584a759e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63caffd640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7596830 0x5584a759e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63caffd640 1 -- 172.21.15.3:0/3500289907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 msgr2=0x5584a759ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63caffd640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a759ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63caffd640 1 -- 172.21.15.3:0/3500289907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a72d9620 con 0x5584a7595f10 2024-05-08T05:55:09.201 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7596830 0x5584a759e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.196+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a759ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63caffd640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7595f10 0x5584a759e140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0df8a0 tx=0x7f63bc0e28e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f254a30ddbeb16c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63aeff5640 1 -- 172.21.15.3:0/3500289907 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc020030 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63bc0a93e0 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63aeff5640 1 -- 172.21.15.3:0/3500289907 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f63bc0a5670 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63aeff5640 1 -- 172.21.15.3:0/3500289907 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f63bc0b2190 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 10 client.25196 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5584a75afbc0 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63aeff5640 1 -- 172.21.15.3:0/3500289907 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc0a93e0 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63aeff5640 1 -- 172.21.15.3:0/3500289907 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f63bc0a3b00 con 0x5584a7595f10 2024-05-08T05:55:09.202 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 10 client.25196 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 10 client.25196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 20 client.25196 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a75b0f10 0x5584a75b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 4 client.25196 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.197+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584a75091e0 con 0x5584a75b0f10 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.198+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a75b0f10 0x5584a75b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.198+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a75b0f10 0x5584a75b3300 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d37c11a01ee561ab server_cookie=712810dab6366af9 in_seq=0 out_seq=0 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.198+0000 7f63aeff5640 10 client.25196 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63aeff5640 1 -- 172.21.15.3:0/3500289907 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5584a75091e0 con 0x5584a75b0f10 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63aeff5640 10 client.25196 handle_command_reply: tid=0 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 client.25196 shutdown 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7595f10 msgr2=0x5584a759e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7595f10 0x5584a759e140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0df8a0 tx=0x7f63bc0e28e0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.203 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 shutdown_connections 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a75b0f10 0x5584a75b3300 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a75a10f0 0x5584a759ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7596830 0x5584a759e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3500289907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7595f10 0x5584a759e140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63aeff5640 0 client.25196 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 >> 172.21.15.3:0/3500289907 conn(0x5584a7592420 msgr2=0x5584a7598150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 shutdown_connections 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.199+0000 7f63d41e2740 1 -- 172.21.15.3:0/3500289907 wait complete. 2024-05-08T05:55:09.204 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.200+0000 7f63d41e2740 20 client.25196 trim_cache size 0 max 16384 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 Processor -- start 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 -- start start 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 0x5584a759e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a759b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a75a10f0 0x5584a759b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1abb40 con 0x5584a7595f10 2024-05-08T05:55:09.217 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1ae3e0 con 0x5584a7596830 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63d41e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1ac1d0 con 0x5584a75a10f0 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63caffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 0x5584a759e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63caffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 0x5584a759e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48060/0 (socket says 172.21.15.3:48060) 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63caffd640 1 -- 172.21.15.3:0/3099994112 learned_addr learned my addr 172.21.15.3:0/3099994112 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.216+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a759b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63caffd640 1 -- 172.21.15.3:0/3099994112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a75a10f0 msgr2=0x5584a759b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63caffd640 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a75a10f0 0x5584a759b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63caffd640 1 -- 172.21.15.3:0/3099994112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 msgr2=0x5584a759b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63caffd640 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a759b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63caffd640 1 -- 172.21.15.3:0/3099994112 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a758dd00 con 0x5584a7595f10 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a759b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63caffd640 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 0x5584a759e9e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0df8d0 tx=0x7f63bc0ad860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba978bc77c829ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:09.218 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63b0ff9640 1 -- 172.21.15.3:0/3099994112 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc020070 con 0x5584a7595f10 2024-05-08T05:55:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63b0ff9640 1 -- 172.21.15.3:0/3099994112 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc009cd0 con 0x5584a7595f10 2024-05-08T05:55:09.219 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.217+0000 7f63b0ff9640 1 -- 172.21.15.3:0/3099994112 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0a8640 con 0x5584a7595f10 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 -- 172.21.15.3:0/3099994112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 msgr2=0x5584a759e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 0x5584a759e9e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f63bc0df8d0 tx=0x7f63bc0ad860 comp rx=0 tx=0).stop 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 -- 172.21.15.3:0/3099994112 shutdown_connections 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a75a10f0 0x5584a759b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a7596830 0x5584a759b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3099994112 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7595f10 0x5584a759e9e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 -- 172.21.15.3:0/3099994112 >> 172.21.15.3:0/3099994112 conn(0x5584a7592420 msgr2=0x5584a75ac920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 -- 172.21.15.3:0/3099994112 shutdown_connections 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 -- 172.21.15.3:0/3099994112 wait complete. 2024-05-08T05:55:09.222 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.218+0000 7f63d41e2740 1 -- start start 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 Processor -- start 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 0x5584a75b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a75a10f0 0x5584a75b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1ac8a0 con 0x5584a7596830 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1add10 con 0x5584a75a10f0 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc1acf70 con 0x5584a7595f10 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63d41e2740 10 client.0 _ll_register_callbacks cb 0x7f63bc0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 0x5584a75b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 0x5584a75b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48064/0 (socket says 172.21.15.3:48064) 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 -- 172.21.15.3:0/3245338851 learned_addr learned my addr 172.21.15.3:0/3245338851 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63caffd640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a75a10f0 0x5584a75b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 -- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 msgr2=0x5584a75b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.223 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 -- 172.21.15.3:0/3245338851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a75a10f0 msgr2=0x5584a75b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.219+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a75a10f0 0x5584a75b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63ca7fc640 1 -- 172.21.15.3:0/3245338851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a75b4350 con 0x5584a7596830 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63cb7fe640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a75a10f0 0x5584a75b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63ca7fc640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 0x5584a75b38d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f63c002a9a0 tx=0x7f63c002ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e3170a5c6fc529 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63aeff5640 1 -- 172.21.15.3:0/3245338851 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c0035040 con 0x5584a7596830 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63aeff5640 1 -- 172.21.15.3:0/3245338851 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f63c0004310 con 0x5584a7596830 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63bc009cd0 con 0x5584a7596830 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63aeff5640 1 -- 172.21.15.3:0/3245338851 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f63c0051050 con 0x5584a7596830 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63d41e2740 10 client.15468 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5584a761b0a0 con 0x5584a7596830 2024-05-08T05:55:09.224 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.220+0000 7f63aeff5640 1 -- 172.21.15.3:0/3245338851 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63bc009cd0 con 0x5584a7596830 2024-05-08T05:55:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63aeff5640 1 -- 172.21.15.3:0/3245338851 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f63c0043ce0 con 0x5584a7596830 2024-05-08T05:55:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63d41e2740 10 client.15468 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63d41e2740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63d41e2740 20 client.15468 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a761b8a0 0x5584a761dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:09.225 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63d41e2740 4 client.15468 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:55:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584a7503050 con 0x5584a761b8a0 2024-05-08T05:55:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.221+0000 7f63caffd640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a761b8a0 0x5584a761dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.222+0000 7f63aeff5640 10 client.15468 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:09.226 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.222+0000 7f63caffd640 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a761b8a0 0x5584a761dcc0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78d4627e9bccf840 server_cookie=59c6340e5fefda60 in_seq=0 out_seq=0 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63aeff5640 1 -- 172.21.15.3:0/3245338851 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584a7503050 con 0x5584a761b8a0 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63aeff5640 10 client.15468 handle_command_reply: tid=0 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 client.15468 shutdown 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 msgr2=0x5584a75b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 0x5584a75b38d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f63c002a9a0 tx=0x7f63c002ac10 comp rx=0 tx=0).stop 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 shutdown_connections 2024-05-08T05:55:09.229 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5584a761b8a0 0x5584a761dcc0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5584a75a10f0 0x5584a75b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5584a7596830 0x5584a75b38d0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 --2- 172.21.15.3:0/3245338851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5584a7595f10 0x5584a75b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63aeff5640 0 client.15468 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.228+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 >> 172.21.15.3:0/3245338851 conn(0x5584a7592420 msgr2=0x5584a759e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 shutdown_connections 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d41e2740 1 -- 172.21.15.3:0/3245338851 wait complete. 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d41e2740 20 client.15468 trim_cache size 0 max 16384 2024-05-08T05:55:09.230 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:55:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:55:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:55:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:55:09.231 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f63b40b2af0 msgr2=0x7f63b40b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f63b40b2af0 0x7f63b40b4fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63c4005c40 tx=0x7f63c4004ab0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc103c20 msgr2=0x7f63cc101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc103c20 0x7f63cc101870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00d6d0 tx=0x7f63bc00db90 comp rx=0 tx=0).stop 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 shutdown_connections 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f63b40b2af0 0x7f63b40b4fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f63cc10cff0 0x7f63cc1022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f63cc104540 0x7f63cc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 --2- 172.21.15.3:0/1267465066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63cc103c20 0x7f63cc101870 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.229+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 >> 172.21.15.3:0/1267465066 conn(0x7f63cc0789d0 msgr2=0x7f63cc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.230+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 shutdown_connections 2024-05-08T05:55:09.232 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:09.230+0000 7f63d14cf640 1 -- 172.21.15.3:0/1267465066 wait complete. 2024-05-08T05:55:09.320 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'adda3b08', quiesce took 6.1 seconds. Will release after: 19.4 2024-05-08T05:55:10.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:09 smithi089 ceph-mon[34495]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:55:10.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:09 smithi104 ceph-mon[34559]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:55:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:09 smithi003 ceph-mon[30137]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:55:12.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:11 smithi089 ceph-mon[34495]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:11 smithi104 ceph-mon[34559]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:12.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:11 smithi003 ceph-mon[30137]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:55:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:13 smithi089 ceph-mon[34495]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-05-08T05:55:14.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:13 smithi104 ceph-mon[34559]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-05-08T05:55:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:13 smithi003 ceph-mon[30137]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-05-08T05:55:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:15 smithi089 ceph-mon[34495]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-08T05:55:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:15 smithi104 ceph-mon[34559]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-08T05:55:16.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:15 smithi003 ceph-mon[30137]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-08T05:55:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:17 smithi089 ceph-mon[34495]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-08T05:55:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:17 smithi104 ceph-mon[34559]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-08T05:55:18.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:17 smithi003 ceph-mon[30137]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-08T05:55:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:19 smithi089 ceph-mon[34495]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-08T05:55:20.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:19 smithi089 ceph-mon[34495]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T05:55:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:19 smithi104 ceph-mon[34559]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-08T05:55:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:19 smithi104 ceph-mon[34559]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T05:55:20.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:19 smithi003 ceph-mon[30137]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 938 B/s wr, 3 op/s 2024-05-08T05:55:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:19 smithi003 ceph-mon[30137]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T05:55:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:21 smithi089 ceph-mon[34495]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 853 B/s wr, 2 op/s 2024-05-08T05:55:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:21 smithi104 ceph-mon[34559]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 853 B/s wr, 2 op/s 2024-05-08T05:55:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:21 smithi003 ceph-mon[30137]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 853 B/s wr, 2 op/s 2024-05-08T05:55:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3671675019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 msgr2=0x7f8c3c104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3671675019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 0x7f8c3c104000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009860 tx=0x7f8c3802a3d0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3671675019 shutdown_connections 2024-05-08T05:55:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3671675019 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 0x7f8c3c10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3671675019 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 0x7f8c3c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.370 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3671675019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 0x7f8c3c104000 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.369+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3671675019 >> 172.21.15.3:0/3671675019 conn(0x7f8c3c0789d0 msgr2=0x7f8c3c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.370+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3671675019 shutdown_connections 2024-05-08T05:55:22.371 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.370+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3671675019 wait complete. 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.371+0000 7f8c43a3f640 1 Processor -- start 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.371+0000 7f8c43a3f640 1 -- start start 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.371+0000 7f8c43a3f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 0x7f8c3c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.371+0000 7f8c43a3f640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 0x7f8c3c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.371+0000 7f8c43a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 0x7f8c3c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.371+0000 7f8c43a3f640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c112c80 con 0x7f8c3c104540 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c43a3f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c112b00 con 0x7f8c3c10cff0 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c43a3f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c112e00 con 0x7f8c3c103c20 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 0x7f8c3c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 0x7f8c3c199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37360/0 (socket says 172.21.15.3:37360) 2024-05-08T05:55:22.373 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 0x7f8c3c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c40fda640 1 -- 172.21.15.3:0/3135613425 learned_addr learned my addr 172.21.15.3:0/3135613425 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c417db640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 0x7f8c3c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 -- 172.21.15.3:0/3135613425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 msgr2=0x7f8c3c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 0x7f8c3c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 -- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 msgr2=0x7f8c3c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 0x7f8c3c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 -- 172.21.15.3:0/3135613425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3c19e210 con 0x7f8c3c10cff0 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.372+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 0x7f8c3c19dae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c34005fd0 tx=0x7f8c3400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.373+0000 7f8c327fc640 1 -- 172.21.15.3:0/3135613425 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34003b70 con 0x7f8c3c10cff0 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.373+0000 7f8c327fc640 1 -- 172.21.15.3:0/3135613425 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c34003d10 con 0x7f8c3c10cff0 2024-05-08T05:55:22.374 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.373+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3c19e500 con 0x7f8c3c10cff0 2024-05-08T05:55:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.373+0000 7f8c327fc640 1 -- 172.21.15.3:0/3135613425 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34004560 con 0x7f8c3c10cff0 2024-05-08T05:55:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.373+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c3c1017a0 con 0x7f8c3c10cff0 2024-05-08T05:55:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.376+0000 7f8c327fc640 1 -- 172.21.15.3:0/3135613425 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8c34022020 con 0x7f8c3c10cff0 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 Processor -- start 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 -- start start 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a6830 0x56132d9b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 0x56132d9b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56132d90b5d0 con 0x56132d9b10f0 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56132d8c4890 con 0x56132d9a5f10 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c46831740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56132d907590 con 0x56132d9a6830 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.377+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 0x56132d9b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 0x56132d9b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37370/0 (socket says 172.21.15.3:37370) 2024-05-08T05:55:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 -- 172.21.15.3:0/2396556205 learned_addr learned my addr 172.21.15.3:0/2396556205 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c417db640 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a6830 0x56132d9b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 -- 172.21.15.3:0/2396556205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a6830 msgr2=0x56132d9b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a6830 0x56132d9b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 -- 172.21.15.3:0/2396556205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 msgr2=0x56132d9a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 -- 172.21.15.3:0/2396556205 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56132d61d210 con 0x56132d9b10f0 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c40fda640 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 0x56132d9b34e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009f10 tx=0x7f8c2c0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23264a4f41188f30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.378+0000 7f8c277fe640 1 -- 172.21.15.3:0/2396556205 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c013d10 con 0x56132d9b10f0 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.379+0000 7f8c277fe640 1 -- 172.21.15.3:0/2396556205 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c2c00f8c0 con 0x56132d9b10f0 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.379+0000 7f8c277fe640 1 -- 172.21.15.3:0/2396556205 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c003850 con 0x56132d9b10f0 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.379+0000 7f8c327fc640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8c280b2950 0x7f8c280b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.379+0000 7f8c417db640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8c280b2950 0x7f8c280b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.382 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.379+0000 7f8c327fc640 1 -- 172.21.15.3:0/3135613425 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8c340e6a90 con 0x7f8c3c10cff0 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 -- 172.21.15.3:0/2396556205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 msgr2=0x56132d9b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 0x56132d9b34e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009f10 tx=0x7f8c2c0095b0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 -- 172.21.15.3:0/2396556205 shutdown_connections 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9b10f0 0x56132d9b34e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a6830 0x56132d9b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 --2- 172.21.15.3:0/2396556205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 -- 172.21.15.3:0/2396556205 >> 172.21.15.3:0/2396556205 conn(0x56132d9a2420 msgr2=0x56132d9a2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c417db640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8c280b2950 0x7f8c280b4e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c3802a9c0 tx=0x7f8c380058b0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 -- 172.21.15.3:0/2396556205 shutdown_connections 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 -- 172.21.15.3:0/2396556205 wait complete. 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.380+0000 7f8c46831740 1 -- start start 2024-05-08T05:55:22.383 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.381+0000 7f8c46831740 1 Processor -- start 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.381+0000 7f8c46831740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.381+0000 7f8c46831740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.381+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.381+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37372/0 (socket says 172.21.15.3:37372) 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c40fda640 1 -- 172.21.15.3:0/222159915 learned_addr learned my addr 172.21.15.3:0/222159915 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132d9aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x56132d9b3d20 con 0x56132d9a6830 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56132d8c8370 con 0x56132d9a5f10 2024-05-08T05:55:22.384 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56132d9b3ea0 con 0x56132d9b10f0 2024-05-08T05:55:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132d9aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.387 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 -- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 msgr2=0x56132d9aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c46831740 10 client.0 _ll_register_callbacks cb 0x56132d61ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132d9aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 -- 172.21.15.3:0/222159915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 msgr2=0x56132d9ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56132d6e9620 con 0x56132d9a5f10 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c40fda640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132d9aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2c00f8c0 con 0x56132d9a5f10 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c417db640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ae050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c38005940 tx=0x7f8c3802a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06c922fee344891 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.382+0000 7f8c24ff9640 10 client.25211 ms_handle_connect on v2:172.21.15.89:3300/0 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c24ff9640 1 -- 172.21.15.3:0/222159915 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380053d0 con 0x56132d9a5f10 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c24ff9640 1 -- 172.21.15.3:0/222159915 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8c38005570 con 0x56132d9a5f10 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c24ff9640 1 -- 172.21.15.3:0/222159915 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c380046c0 con 0x56132d9a5f10 2024-05-08T05:55:22.388 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c46831740 10 client.25211 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56132d9bfab0 con 0x56132d9a5f10 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c24ff9640 1 -- 172.21.15.3:0/222159915 <== mon.2 v2:172.21.15.89:3300/0 4 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8c38004860 con 0x56132d9a5f10 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.383+0000 7f8c46831740 10 client.25211 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c46831740 10 client.25211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c46831740 20 client.25211 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132d9c12a0 0x56132d9c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c46831740 4 client.25211 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56132d9191e0 con 0x56132d9c12a0 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c40fda640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132d9c12a0 0x56132d9c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c40fda640 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132d9c12a0 0x56132d9c3760 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c470a09a6b08e6 server_cookie=2f9e31a137206fae in_seq=0 out_seq=0 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.384+0000 7f8c24ff9640 10 client.25211 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:22.389 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c24ff9640 1 -- 172.21.15.3:0/222159915 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56132d9191e0 con 0x56132d9c12a0 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c24ff9640 10 client.25211 handle_command_reply: tid=0 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 client.25211 shutdown 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 msgr2=0x56132d9ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ae050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8c38005940 tx=0x7f8c3802a3d0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 shutdown_connections 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132d9c12a0 0x56132d9c3760 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132d9aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 --2- 172.21.15.3:0/222159915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ae050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 >> 172.21.15.3:0/222159915 conn(0x56132d9a2420 msgr2=0x56132d9a8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c24ff9640 0 client.25211 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.385+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 shutdown_connections 2024-05-08T05:55:22.390 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.386+0000 7f8c46831740 1 -- 172.21.15.3:0/222159915 wait complete. 2024-05-08T05:55:22.391 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.386+0000 7f8c46831740 20 client.25211 trim_cache size 0 max 16384 2024-05-08T05:55:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.416+0000 7f8c46831740 1 Processor -- start 2024-05-08T05:55:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.416+0000 7f8c46831740 1 -- start start 2024-05-08T05:55:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c46831740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a5f10 0x56132d9ac940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c46831740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.417 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9a88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37378/0 (socket says 172.21.15.3:37378) 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c40fda640 1 -- 172.21.15.3:0/823274793 learned_addr learned my addr 172.21.15.3:0/823274793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c46831740 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9b10f0 0x56132d9ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c417db640 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a5f10 0x56132d9ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1aba80 con 0x56132d9a6830 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ae320 con 0x56132d9b10f0 2024-05-08T05:55:22.418 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ac110 con 0x56132d9a5f10 2024-05-08T05:55:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9b10f0 0x56132d9ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c417db640 1 -- 172.21.15.3:0/823274793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9b10f0 msgr2=0x56132d9ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c417db640 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9b10f0 0x56132d9ae8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c417db640 1 -- 172.21.15.3:0/823274793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 msgr2=0x56132d9a88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c417db640 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9a88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c417db640 1 -- 172.21.15.3:0/823274793 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56132d9ad020 con 0x56132d9a5f10 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.417+0000 7f8c40fda640 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9a88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.418+0000 7f8c417db640 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a5f10 0x56132d9ac940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c380095f0 tx=0x7f8c3804b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b5200d21c4b048 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.418+0000 7f8c26ffd640 1 -- 172.21.15.3:0/823274793 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c380303a0 con 0x56132d9a5f10 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.418+0000 7f8c26ffd640 1 -- 172.21.15.3:0/823274793 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c2c00f8c0 con 0x56132d9a5f10 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.418+0000 7f8c26ffd640 1 -- 172.21.15.3:0/823274793 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38052920 con 0x56132d9a5f10 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a5f10 msgr2=0x56132d9ac940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a5f10 0x56132d9ac940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8c380095f0 tx=0x7f8c3804b000 comp rx=0 tx=0).stop 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 shutdown_connections 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9b10f0 0x56132d9ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9a88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 --2- 172.21.15.3:0/823274793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9a5f10 0x56132d9ac940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 >> 172.21.15.3:0/823274793 conn(0x56132d9a2420 msgr2=0x56132d9b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 shutdown_connections 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 -- 172.21.15.3:0/823274793 wait complete. 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.419+0000 7f8c46831740 1 -- start start 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.420+0000 7f8c46831740 1 Processor -- start 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.420+0000 7f8c46831740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.420+0000 7f8c46831740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.420+0000 7f8c46831740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132da2ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c417db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:37380/0 (socket says 172.21.15.3:37380) 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c417db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ab220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:36838/0 (socket says 172.21.15.3:36838) 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 -- 172.21.15.3:0/1473565391 learned_addr learned my addr 172.21.15.3:0/1473565391 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1ac7e0 con 0x56132d9a6830 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1adc50 con 0x56132d9a5f10 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3c1aceb0 con 0x56132d9b10f0 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c41fdc640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132da2ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c46831740 10 client.0 _ll_register_callbacks cb 0x7f8c340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 -- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 msgr2=0x56132da2ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132da2ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 -- 172.21.15.3:0/1473565391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 msgr2=0x56132d9ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56132d9ac030 con 0x56132d9a6830 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c417db640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c2c00f8c0 con 0x56132d9a6830 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c40fda640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ab760 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009f10 tx=0x7f8c2c009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94614620ab034aa3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.421+0000 7f8c24ff9640 10 client.15486 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.422+0000 7f8c24ff9640 1 -- 172.21.15.3:0/1473565391 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c031060 con 0x56132d9a6830 2024-05-08T05:55:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.422+0000 7f8c24ff9640 1 -- 172.21.15.3:0/1473565391 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8c2c0052c0 con 0x56132d9a6830 2024-05-08T05:55:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.422+0000 7f8c24ff9640 1 -- 172.21.15.3:0/1473565391 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c2c0054c0 con 0x56132d9a6830 2024-05-08T05:55:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.422+0000 7f8c46831740 10 client.15486 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:22.430 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.422+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56132da2c0f0 con 0x56132d9a6830 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.422+0000 7f8c24ff9640 1 -- 172.21.15.3:0/1473565391 <== mon.0 v2:172.21.15.3:3300/0 4 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f8c2c0035f0 con 0x56132d9a6830 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c46831740 10 client.15486 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c46831740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c46831740 20 client.15486 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c46831740 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132da2c510 0x56132da2e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c46831740 4 client.15486 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56132d913050 con 0x56132da2c510 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.423+0000 7f8c417db640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132da2c510 0x56132da2e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.424+0000 7f8c24ff9640 10 client.15486 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.424+0000 7f8c417db640 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132da2c510 0x56132da2e900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=268ec5f1b4907728 server_cookie=4539e401a269fa8a in_seq=0 out_seq=0 2024-05-08T05:55:22.434 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c24ff9640 1 -- 172.21.15.3:0/1473565391 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56132d913050 con 0x56132da2c510 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c24ff9640 10 client.15486 handle_command_reply: tid=0 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 client.15486 shutdown 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 msgr2=0x56132d9ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ab760 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009f10 tx=0x7f8c2c009a80 comp rx=0 tx=0).stop 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 shutdown_connections 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x56132da2c510 0x56132da2e900 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132d9b10f0 0x56132da2ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c24ff9640 0 client.15486 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x56132d9a6830 0x56132d9ab760 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.435 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 --2- 172.21.15.3:0/1473565391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56132d9a5f10 0x56132d9ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.429+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 >> 172.21.15.3:0/1473565391 conn(0x56132d9a2420 msgr2=0x56132d9a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.430+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 shutdown_connections 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.430+0000 7f8c46831740 1 -- 172.21.15.3:0/1473565391 wait complete. 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.430+0000 7f8c46831740 20 client.15486 trim_cache size 0 max 16384 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:55:22.436 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:55:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8c280b2950 msgr2=0x7f8c280b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8c280b2950 0x7f8c280b4e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8c3802a9c0 tx=0x7f8c380058b0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.437 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 msgr2=0x7f8c3c19dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 0x7f8c3c19dae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8c34005fd0 tx=0x7f8c3400c6a0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 shutdown_connections 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8c280b2950 0x7f8c280b4e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8c3c10cff0 0x7f8c3c19dae0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8c3c104540 0x7f8c3c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 --2- 172.21.15.3:0/3135613425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c3c103c20 0x7f8c3c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 >> 172.21.15.3:0/3135613425 conn(0x7f8c3c0789d0 msgr2=0x7f8c3c0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 shutdown_connections 2024-05-08T05:55:22.438 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:22.431+0000 7f8c43a3f640 1 -- 172.21.15.3:0/3135613425 wait complete. 2024-05-08T05:55:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:23 smithi089 ceph-mon[34495]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-08T05:55:24.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:23 smithi104 ceph-mon[34559]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-08T05:55:24.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:23 smithi003 ceph-mon[30137]: pgmap v52: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-05-08T05:55:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:25 smithi089 ceph-mon[34495]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-05-08T05:55:26.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:25 smithi104 ceph-mon[34559]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-05-08T05:55:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:25 smithi003 ceph-mon[30137]: pgmap v53: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-05-08T05:55:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:27 smithi089 ceph-mon[34495]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-05-08T05:55:28.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:27 smithi104 ceph-mon[34559]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-05-08T05:55:28.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:27 smithi003 ceph-mon[30137]: pgmap v54: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-05-08T05:55:28.721 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:55:29.118 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:29 smithi003 ceph-mon[30137]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-08T05:55:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:29 smithi089 ceph-mon[34495]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-08T05:55:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:29 smithi104 ceph-mon[34559]: pgmap v55: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 511 B/s wr, 3 op/s 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 -- 172.21.15.3:0/3473945554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8102d60 msgr2=0x7f0ea810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/3473945554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8102d60 0x7f0ea810f250 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000b0a0 tx=0x7f0ea000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 -- 172.21.15.3:0/3473945554 shutdown_connections 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/3473945554 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8102d60 0x7f0ea810f250 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/3473945554 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea8108410 0x7f0ea81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/3473945554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ea81023a0 0x7f0ea8102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.147 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.146+0000 7f0ead2ad700 1 -- 172.21.15.3:0/3473945554 >> 172.21.15.3:0/3473945554 conn(0x7f0ea8079350 msgr2=0x7f0ea8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.147+0000 7f0ead2ad700 1 -- 172.21.15.3:0/3473945554 shutdown_connections 2024-05-08T05:55:31.148 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.147+0000 7f0ead2ad700 1 -- 172.21.15.3:0/3473945554 wait complete. 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.147+0000 7f0ead2ad700 1 Processor -- start 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 -- start start 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea81023a0 0x7f0ea819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ea8102d60 0x7f0ea819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 0x7f0ea8197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8111470 con 0x7f0ea8108410 2024-05-08T05:55:31.150 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81112f0 con 0x7f0ea81023a0 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ead2ad700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea81115f0 con 0x7f0ea8102d60 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ea77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 0x7f0ea8197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ea67fc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ea8102d60 0x7f0ea819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ea6ffd700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea81023a0 0x7f0ea819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ea77fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 0x7f0ea8197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:46248/0 (socket says 172.21.15.3:46248) 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.148+0000 7f0ea77fe700 1 -- 172.21.15.3:0/2280885299 learned_addr learned my addr 172.21.15.3:0/2280885299 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ea77fe700 1 -- 172.21.15.3:0/2280885299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ea8102d60 msgr2=0x7f0ea819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ea77fe700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ea8102d60 0x7f0ea819dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ea77fe700 1 -- 172.21.15.3:0/2280885299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea81023a0 msgr2=0x7f0ea819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ea77fe700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea81023a0 0x7f0ea819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ea77fe700 1 -- 172.21.15.3:0/2280885299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea8197f50 con 0x7f0ea8108410 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ea77fe700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 0x7f0ea8197670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000b830 tx=0x7f0ea0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0021070 con 0x7f0ea8108410 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ea0007e30 con 0x7f0ea8108410 2024-05-08T05:55:31.151 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0004700 con 0x7f0ea8108410 2024-05-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.149+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea81981e0 con 0x7f0ea8108410 2024-05-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.150+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81a4520 con 0x7f0ea8108410 2024-05-08T05:55:31.152 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.151+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea8103ac0 con 0x7f0ea8108410 2024-05-08T05:55:31.153 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.153+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f0ea0007980 con 0x7f0ea8108410 2024-05-08T05:55:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.154+0000 7f0e8ffff700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0e900b2a70 0x7f0e900b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.154+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0ea00e32b0 con 0x7f0ea8108410 2024-05-08T05:55:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.155+0000 7f0ea6ffd700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0e900b2a70 0x7f0e900b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:31.155 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.155+0000 7f0ea6ffd700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0e900b2a70 0x7f0e900b4f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800ce30 tx=0x7f0e9800e040 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:31.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.163+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ea00aa9b0 con 0x7f0ea8108410 2024-05-08T05:55:31.546 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.546+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ea806a1c0 con 0x7f0ea8108410 2024-05-08T05:55:31.547 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.547+0000 7f0e8ffff700 1 -- 172.21.15.3:0/2280885299 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7f0ea806a1c0 con 0x7f0ea8108410 2024-05-08T05:55:31.547 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:55:31.548 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:55:31.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.550+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0e900b2a70 msgr2=0x7f0e900b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:31.550 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.550+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0e900b2a70 0x7f0e900b4f30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800ce30 tx=0x7f0e9800e040 comp rx=0 tx=0).stop 2024-05-08T05:55:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.550+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 msgr2=0x7f0ea8197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.550+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 0x7f0ea8197670 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0ea000b830 tx=0x7f0ea0004ce0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 shutdown_connections 2024-05-08T05:55:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0ea8108410 0x7f0ea8197670 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0ea81023a0 0x7f0ea819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.551 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f0e900b2a70 0x7f0e900b4f30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 --2- 172.21.15.3:0/2280885299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ea8102d60 0x7f0ea819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:31.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 >> 172.21.15.3:0/2280885299 conn(0x7f0ea8079350 msgr2=0x7f0ea8100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:31.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 shutdown_connections 2024-05-08T05:55:31.552 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:31.551+0000 7f0ead2ad700 1 -- 172.21.15.3:0/2280885299 wait complete. 2024-05-08T05:55:31.552 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T05:55:31.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:31 smithi003 ceph-mon[30137]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-05-08T05:55:31.884 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:31 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2280885299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:31 smithi089 ceph-mon[34495]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-05-08T05:55:32.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:31 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2280885299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:31 smithi104 ceph-mon[34559]: pgmap v56: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-05-08T05:55:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:31 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2280885299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:55:32.205 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24493 quiesce db --set-id adda3b08 --release --await' 2024-05-08T05:55:32.205 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24493 quiesce db --set-id adda3b08 --release --await 2024-05-08T05:55:32.573 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:55:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:33 smithi089 ceph-mon[34495]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-05-08T05:55:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:33 smithi104 ceph-mon[34559]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-05-08T05:55:34.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:33 smithi003 ceph-mon[30137]: pgmap v57: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 341 B/s wr, 3 op/s 2024-05-08T05:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 -- 172.21.15.3:0/554477365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc34102d60 msgr2=0x7efc3410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 --2- 172.21.15.3:0/554477365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc34102d60 0x7efc3410f250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00b0a0 tx=0x7efc2c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.518 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 -- 172.21.15.3:0/554477365 shutdown_connections 2024-05-08T05:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 --2- 172.21.15.3:0/554477365 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc34102d60 0x7efc3410f250 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 --2- 172.21.15.3:0/554477365 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34108410 0x7efc341087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 --2- 172.21.15.3:0/554477365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc341023a0 0x7efc34102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.519 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.518+0000 7efc3b063700 1 -- 172.21.15.3:0/554477365 >> 172.21.15.3:0/554477365 conn(0x7efc34079350 msgr2=0x7efc34079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.519+0000 7efc3b063700 1 -- 172.21.15.3:0/554477365 shutdown_connections 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.519+0000 7efc3b063700 1 -- 172.21.15.3:0/554477365 wait complete. 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.519+0000 7efc3b063700 1 Processor -- start 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 -- start start 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 0x7efc3410fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34102d60 0x7efc34110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.520 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34108410 0x7efc34110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc38dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 0x7efc3410fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc38dff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 0x7efc3410fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36024/0 (socket says 172.21.15.3:36024) 2024-05-08T05:55:34.521 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc38dff700 1 -- 172.21.15.3:0/3739715610 learned_addr learned my addr 172.21.15.3:0/3739715610 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc34113650 con 0x7efc341023a0 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc341134d0 con 0x7efc34102d60 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc341137d0 con 0x7efc34108410 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc33fff700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34102d60 0x7efc34110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.520+0000 7efc39600700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34108410 0x7efc34110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc38dff700 1 -- 172.21.15.3:0/3739715610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34108410 msgr2=0x7efc34110760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc38dff700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34108410 0x7efc34110760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc38dff700 1 -- 172.21.15.3:0/3739715610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34102d60 msgr2=0x7efc34110220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc38dff700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34102d60 0x7efc34110220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.522 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc38dff700 1 -- 172.21.15.3:0/3739715610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc341a8050 con 0x7efc341023a0 2024-05-08T05:55:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc38dff700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 0x7efc3410fce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c9b0 tx=0x7efc2800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc31ffb700 1 -- 172.21.15.3:0/3739715610 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28007b00 con 0x7efc341023a0 2024-05-08T05:55:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.521+0000 7efc31ffb700 1 -- 172.21.15.3:0/3739715610 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc28007ca0 con 0x7efc341023a0 2024-05-08T05:55:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.522+0000 7efc31ffb700 1 -- 172.21.15.3:0/3739715610 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28018730 con 0x7efc341023a0 2024-05-08T05:55:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.522+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc341a8340 con 0x7efc341023a0 2024-05-08T05:55:34.523 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.522+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a88d0 con 0x7efc341023a0 2024-05-08T05:55:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.526+0000 7efc31ffb700 1 -- 172.21.15.3:0/3739715610 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7efc2800f450 con 0x7efc341023a0 2024-05-08T05:55:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.526+0000 7efc458f6b80 1 Processor -- start 2024-05-08T05:55:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.526+0000 7efc458f6b80 1 -- start start 2024-05-08T05:55:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d23d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 0x5572c0d1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.527 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d1ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc458f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5572c0c67960 con 0x5572c0d23920 2024-05-08T05:55:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc458f6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5572c0c068c0 con 0x5572c0d24240 2024-05-08T05:55:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc458f6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5572c0c63920 con 0x5572c0d1e5d0 2024-05-08T05:55:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc38dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 0x5572c0d1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc38dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 0x5572c0d1dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:58030/0 (socket says 172.21.15.3:58030) 2024-05-08T05:55:34.528 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc38dff700 1 -- 172.21.15.3:0/1668725462 learned_addr learned my addr 172.21.15.3:0/1668725462 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc31ffb700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7efc240b2a70 0x7efc240b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.527+0000 7efc33fff700 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d1ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc31ffb700 1 -- 172.21.15.3:0/3739715610 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efc280e27e0 con 0x7efc341023a0 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc38dff700 1 -- 172.21.15.3:0/1668725462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 msgr2=0x5572c0d1ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc38dff700 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d1ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc38dff700 1 -- 172.21.15.3:0/1668725462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 msgr2=0x5572c0d23d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.529 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc38dff700 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d23d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc38dff700 1 -- 172.21.15.3:0/1668725462 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c0bb49d0 con 0x5572c0d24240 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc38dff700 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 0x5572c0d1dfb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc280dfb00 tx=0x7efc28000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775d725d528aaa36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc39600700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7efc240b2a70 0x7efc240b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc1effd700 1 -- 172.21.15.3:0/1668725462 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc280a8ec0 con 0x5572c0d24240 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc1effd700 1 -- 172.21.15.3:0/1668725462 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc280a9260 con 0x5572c0d24240 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.528+0000 7efc1effd700 1 -- 172.21.15.3:0/1668725462 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc28016de0 con 0x5572c0d24240 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 -- 172.21.15.3:0/1668725462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 msgr2=0x5572c0d1dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.530 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 0x5572c0d1dfb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efc280dfb00 tx=0x7efc28000bc0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.534 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc39600700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7efc240b2a70 0x7efc240b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0062a0 tx=0x7efc2c006210 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 -- 172.21.15.3:0/1668725462 shutdown_connections 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d1ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d23d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1668725462 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d24240 0x5572c0d1dfb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.529+0000 7efc458f6b80 1 -- 172.21.15.3:0/1668725462 >> 172.21.15.3:0/1668725462 conn(0x5572c0c41200 msgr2=0x5572c0d0ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.530+0000 7efc458f6b80 1 -- 172.21.15.3:0/1668725462 shutdown_connections 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.530+0000 7efc458f6b80 1 -- 172.21.15.3:0/1668725462 wait complete. 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.530+0000 7efc458f6b80 1 -- start start 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.530+0000 7efc458f6b80 1 Processor -- start 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 0x5572c0d2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.535 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d24240 0x5572c0d278e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5572c0d2acb0 con 0x5572c0d23920 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5572c0bcdd30 con 0x5572c0d1e5d0 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5572c0d2ae30 con 0x5572c0d24240 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc458f6b80 10 client.0 _ll_register_callbacks cb 0x5572c0bffa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc38dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 0x5572c0d2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc38dff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 0x5572c0d2a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:58046/0 (socket says 172.21.15.3:58046) 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc38dff700 1 -- 172.21.15.3:0/1747004703 learned_addr learned my addr 172.21.15.3:0/1747004703 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.531+0000 7efc33fff700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc39600700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d24240 0x5572c0d278e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc38dff700 1 -- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d24240 msgr2=0x5572c0d278e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.536 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc38dff700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d24240 0x5572c0d278e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc38dff700 1 -- 172.21.15.3:0/1747004703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 msgr2=0x5572c0d273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc38dff700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d273a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc38dff700 1 -- 172.21.15.3:0/1747004703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c0c03790 con 0x5572c0d1e5d0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc39600700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d24240 0x5572c0d278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc38dff700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 0x5572c0d2a000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc280a57d0 tx=0x7efc280a8f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a53a962cbf5488 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc13fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc13fff700 1 -- 172.21.15.3:0/1747004703 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc280a9e50 con 0x5572c0d1e5d0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.532+0000 7efc13fff700 1 -- 172.21.15.3:0/1747004703 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efc280e5e00 con 0x5572c0d1e5d0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc280a9260 con 0x5572c0d1e5d0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc13fff700 1 -- 172.21.15.3:0/1747004703 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc280b25b0 con 0x5572c0d1e5d0 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 10 client.25226 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:34.537 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5572c0d2acb0 con 0x5572c0d1e5d0 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc13fff700 1 -- 172.21.15.3:0/1747004703 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc280e2400 con 0x5572c0d1e5d0 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc13fff700 1 -- 172.21.15.3:0/1747004703 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7efc28020020 con 0x5572c0d1e5d0 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 10 client.25226 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 10 client.25226 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 20 client.25226 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0d378f0 0x5572c0d39ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 4 client.25226 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.533+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efc280a9260 con 0x5572c0d378f0 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.534+0000 7efc33fff700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0d378f0 0x5572c0d39ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.534+0000 7efc13fff700 10 client.25226 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:34.538 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.534+0000 7efc33fff700 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0d378f0 0x5572c0d39ce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dde361177e747e8 server_cookie=9613e2350e82e7ce in_seq=0 out_seq=0 2024-05-08T05:55:34.539 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc13fff700 1 -- 172.21.15.3:0/1747004703 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7efc280a9260 con 0x5572c0d378f0 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc13fff700 10 client.25226 handle_command_reply: tid=0 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 client.25226 shutdown 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 msgr2=0x5572c0d2a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 0x5572c0d2a000 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc280a57d0 tx=0x7efc280a8f30 comp rx=0 tx=0).stop 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 shutdown_connections 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d24240 0x5572c0d278e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0d378f0 0x5572c0d39ce0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d1e5d0 0x5572c0d2a000 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 --2- 172.21.15.3:0/1747004703 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d23920 0x5572c0d273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc13fff700 0 client.25226 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 >> 172.21.15.3:0/1747004703 conn(0x5572c0c41200 msgr2=0x5572c0d22d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:34.540 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.535+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 shutdown_connections 2024-05-08T05:55:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.536+0000 7efc458f6b80 1 -- 172.21.15.3:0/1747004703 wait complete. 2024-05-08T05:55:34.541 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.536+0000 7efc458f6b80 20 client.25226 trim_cache size 0 max 16384 2024-05-08T05:55:34.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 Processor -- start 2024-05-08T05:55:34.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 -- start start 2024-05-08T05:55:34.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d33da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d263c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.558 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc341ab190 con 0x5572c0d24240 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc341ada30 con 0x5572c0d23920 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.557+0000 7efc458f6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc341ab820 con 0x5572c0d1e5d0 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d33da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d33da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.3:60496/0 (socket says 172.21.15.3:60496) 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc39600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 -- 172.21.15.3:0/2523067345 learned_addr learned my addr 172.21.15.3:0/2523067345 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc33fff700 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d263c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 -- 172.21.15.3:0/2523067345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 msgr2=0x5572c0d263c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d263c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 -- 172.21.15.3:0/2523067345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 msgr2=0x5572c0d203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.559 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d203f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 -- 172.21.15.3:0/2523067345 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c0d0af40 con 0x5572c0d1e5d0 2024-05-08T05:55:34.560 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc39600700 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.558+0000 7efc38dff700 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d33da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cfd0 tx=0x7efc280a8ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f7883784421eba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.559+0000 7efc1e7fc700 1 -- 172.21.15.3:0/2523067345 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc280e2a60 con 0x5572c0d1e5d0 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.559+0000 7efc1e7fc700 1 -- 172.21.15.3:0/2523067345 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc280a9260 con 0x5572c0d1e5d0 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.559+0000 7efc1e7fc700 1 -- 172.21.15.3:0/2523067345 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc280a99b0 con 0x5572c0d1e5d0 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 -- 172.21.15.3:0/2523067345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 msgr2=0x5572c0d33da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d33da0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cfd0 tx=0x7efc280a8ec0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 -- 172.21.15.3:0/2523067345 shutdown_connections 2024-05-08T05:55:34.563 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d33da0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 --2- 172.21.15.3:0/2523067345 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d263c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 -- 172.21.15.3:0/2523067345 >> 172.21.15.3:0/2523067345 conn(0x5572c0c41200 msgr2=0x5572c0d25cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 -- 172.21.15.3:0/2523067345 shutdown_connections 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 -- 172.21.15.3:0/2523067345 wait complete. 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.560+0000 7efc458f6b80 1 -- start start 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 Processor -- start 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d30500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d2d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d2d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.564 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7efc341abef0 con 0x5572c0d24240 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efc341ad360 con 0x5572c0d23920 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc341ac5c0 con 0x5572c0d1e5d0 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc458f6b80 10 client.0 _ll_register_callbacks cb 0x5572c0bffa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.561+0000 7efc39600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d2d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d2d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc39600700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d2d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36052/0 (socket says 172.21.15.3:36052) 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc39600700 1 -- 172.21.15.3:0/4019393808 learned_addr learned my addr 172.21.15.3:0/4019393808 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 -- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 msgr2=0x5572c0d30500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc38dff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d30500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d30500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.565 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 -- 172.21.15.3:0/4019393808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 msgr2=0x5572c0d2d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d2d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 -- 172.21.15.3:0/4019393808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c0d2dcb0 con 0x5572c0d23920 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc39600700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d2d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc38dff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d30500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc33fff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d2d0f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5572c0d28970 tx=0x7efc20011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3d62e25d5bf066 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.562+0000 7efc13fff700 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc13fff700 1 -- 172.21.15.3:0/4019393808 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2001a490 con 0x5572c0d23920 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc280a9260 con 0x5572c0d23920 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc13fff700 1 -- 172.21.15.3:0/4019393808 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efc2001a630 con 0x5572c0d23920 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc13fff700 1 -- 172.21.15.3:0/4019393808 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efc20011e30 con 0x5572c0d23920 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc458f6b80 10 client.25232 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:34.566 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5572c0da2e60 con 0x5572c0d23920 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.563+0000 7efc13fff700 1 -- 172.21.15.3:0/4019393808 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc280a9260 con 0x5572c0d23920 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc13fff700 1 -- 172.21.15.3:0/4019393808 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7efc200103d0 con 0x5572c0d23920 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc458f6b80 10 client.25232 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc458f6b80 10 client.25232 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc458f6b80 20 client.25232 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc458f6b80 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0da3520 0x5572c0da5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc458f6b80 4 client.25232 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "adda3b08", "release": true, "await": true}] 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "quiesce db", "set_id": "adda3b08", "release": true, "await": true}) -- 0x5572c0ce1750 con 0x5572c0da3520 2024-05-08T05:55:34.569 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.564+0000 7efc38dff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0da3520 0x5572c0da5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc38dff700 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0da3520 0x5572c0da5940 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef9432c016f5d54 server_cookie=2aa27ca4f8c125ba in_seq=0 out_seq=0 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc13fff700 10 client.25232 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc13fff700 1 -- 172.21.15.3:0/4019393808 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5572c0ce1750 con 0x5572c0da3520 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc13fff700 10 client.25232 handle_command_reply: tid=0 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc458f6b80 1 client.25232 shutdown 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 msgr2=0x5572c0d2d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.565+0000 7efc458f6b80 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d2d0f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x5572c0d28970 tx=0x7efc20011720 comp rx=0 tx=0).stop 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 shutdown_connections 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5572c0d24240 0x5572c0d2d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572c0d1e5d0 0x5572c0d30500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5572c0da3520 0x5572c0da5940 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.570 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 --2- 172.21.15.3:0/4019393808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5572c0d23920 0x5572c0d2d0f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc13fff700 0 client.25232 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 >> 172.21.15.3:0/4019393808 conn(0x5572c0c41200 msgr2=0x5572c0d289a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 shutdown_connections 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 1 -- 172.21.15.3:0/4019393808 wait complete. 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.566+0000 7efc458f6b80 20 client.25232 trim_cache size 0 max 16384 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 23, 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24493, 2024-05-08T05:55:34.571 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 8, 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": { 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "adda3b08": { 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "version": 8, 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "age_ref": 0.0, 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "timeout": 60.0, 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "expiration": 79.4, 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "members": { 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "file:/": { 2024-05-08T05:55:34.572 INFO:teuthology.orchestra.run.smithi003.stdout: "excluded": false, 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: "state": { 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RELEASED", 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: "age": 0.0 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T05:55:34.573 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:55:34.574 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.567+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7efc240b2a70 msgr2=0x7efc240b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.567+0000 7efc3b063700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7efc240b2a70 0x7efc240b4f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0062a0 tx=0x7efc2c006210 comp rx=0 tx=0).stop 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 msgr2=0x7efc3410fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 0x7efc3410fce0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c9b0 tx=0x7efc2800cd70 comp rx=0 tx=0).stop 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 shutdown_connections 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7efc240b2a70 0x7efc240b4f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc34108410 0x7efc34110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7efc341023a0 0x7efc3410fce0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 --2- 172.21.15.3:0/3739715610 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efc34102d60 0x7efc34110220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 >> 172.21.15.3:0/3739715610 conn(0x7efc34079350 msgr2=0x7efc34100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 shutdown_connections 2024-05-08T05:55:34.575 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:55:34.568+0000 7efc3b063700 1 -- 172.21.15.3:0/3739715610 wait complete. 2024-05-08T05:55:35.386 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'adda3b08', seconds elapsed: 32.1 2024-05-08T05:55:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 -- 172.21.15.3:0/716710468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 msgr2=0x7f1d9c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/716710468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c101d10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009930 tx=0x7f1d8802a420 comp rx=0 tx=0).stop 2024-05-08T05:55:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 -- 172.21.15.3:0/716710468 shutdown_connections 2024-05-08T05:55:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/716710468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c102250 0x7f1d9c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.530 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/716710468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c101d10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/716710468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d9c1078a0 0x7f1d9c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 -- 172.21.15.3:0/716710468 >> 172.21.15.3:0/716710468 conn(0x7f1d9c0789d0 msgr2=0x7f1d9c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 -- 172.21.15.3:0/716710468 shutdown_connections 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 -- 172.21.15.3:0/716710468 wait complete. 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 Processor -- start 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.528+0000 7f1da2fdd640 1 -- start start 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d9c102250 0x7f1d9c199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c1078a0 0x7f1d9c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c111b60 con 0x7f1d9c101890 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1119e0 con 0x7f1d9c1078a0 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c111ce0 con 0x7f1d9c102250 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.531 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36076/0 (socket says 172.21.15.3:36076) 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 -- 172.21.15.3:0/1574256992 learned_addr learned my addr 172.21.15.3:0/1574256992 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da157a640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c1078a0 0x7f1d9c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d9c102250 0x7f1d9c199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 -- 172.21.15.3:0/1574256992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d9c102250 msgr2=0x7f1d9c199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d9c102250 0x7f1d9c199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 -- 172.21.15.3:0/1574256992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c1078a0 msgr2=0x7f1d9c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c1078a0 0x7f1d9c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 -- 172.21.15.3:0/1574256992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d9c19e180 con 0x7f1d9c101890 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da157a640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c1078a0 0x7f1d9c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da0d79640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c199230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000d770 tx=0x7f1d9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1d99ffb640 1 -- 172.21.15.3:0/1574256992 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90017070 con 0x7f1d9c101890 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d9c19e470 con 0x7f1d9c101890 2024-05-08T05:55:35.532 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.529+0000 7f1d99ffb640 1 -- 172.21.15.3:0/1574256992 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d900054d0 con 0x7f1d9c101890 2024-05-08T05:55:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.530+0000 7f1d99ffb640 1 -- 172.21.15.3:0/1574256992 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90004c30 con 0x7f1d9c101890 2024-05-08T05:55:35.533 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.530+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d9c1a5d00 con 0x7f1d9c101890 2024-05-08T05:55:35.539 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.532+0000 7f1d99ffb640 1 -- 172.21.15.3:0/1574256992 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1d90004e10 con 0x7f1d9c101890 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.532+0000 7f1da5dd3740 1 Processor -- start 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.532+0000 7f1da5dd3740 1 -- start start 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1565e10 0x563be15661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1574fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 0x563be15778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563be14cb5d0 con 0x563be1575500 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563be1484890 con 0x563be1565e10 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563be14c7590 con 0x563be1566730 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 0x563be15778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 0x563be15778f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36090/0 (socket says 172.21.15.3:36090) 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 -- 172.21.15.3:0/181593907 learned_addr learned my addr 172.21.15.3:0/181593907 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da157a640 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1565e10 0x563be15661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.540 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da0d79640 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1574fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d99ffb640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1d8c0b2950 0x7f1d8c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 -- 172.21.15.3:0/181593907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 msgr2=0x563be1574fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1574fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 -- 172.21.15.3:0/181593907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1565e10 msgr2=0x563be15661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1565e10 0x563be15661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1d9bfff640 1 -- 172.21.15.3:0/181593907 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563be11dd210 con 0x563be1575500 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da0d79640 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.533+0000 7f1da157a640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1d8c0b2950 0x7f1d8c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.534+0000 7f1d99ffb640 1 -- 172.21.15.3:0/1574256992 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1d900e23e0 con 0x7f1d9c101890 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.534+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 0x563be15778f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802a940 tx=0x7f1d88031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b334d351d8a5a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.534+0000 7f1da157a640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1d8c0b2950 0x7f1d8c0b4e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005fd0 tx=0x7f1d94004a40 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.534+0000 7f1d86ffd640 1 -- 172.21.15.3:0/181593907 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88040070 con 0x563be1575500 2024-05-08T05:55:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.534+0000 7f1d86ffd640 1 -- 172.21.15.3:0/181593907 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d880316a0 con 0x563be1575500 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.534+0000 7f1d86ffd640 1 -- 172.21.15.3:0/181593907 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88035040 con 0x563be1575500 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 -- 172.21.15.3:0/181593907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 msgr2=0x563be15778f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 0x563be15778f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802a940 tx=0x7f1d88031bc0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 -- 172.21.15.3:0/181593907 shutdown_connections 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1575500 0x563be15778f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/181593907 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1565e10 0x563be15661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 -- 172.21.15.3:0/181593907 >> 172.21.15.3:0/181593907 conn(0x563be1562420 msgr2=0x563be1562800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 -- 172.21.15.3:0/181593907 shutdown_connections 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 -- 172.21.15.3:0/181593907 wait complete. 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 -- start start 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.535+0000 7f1da5dd3740 1 Processor -- start 2024-05-08T05:55:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1572310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1572850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be1572d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x563be15780d0 con 0x563be1565e10 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563be1488370 con 0x563be1575500 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563be1578250 con 0x563be1566730 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da5dd3740 10 client.0 _ll_register_callbacks cb 0x563be11dca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1572310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1572310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36096/0 (socket says 172.21.15.3:36096) 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 -- 172.21.15.3:0/2075905785 learned_addr learned my addr 172.21.15.3:0/2075905785 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1572850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da157a640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be1572d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 -- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 msgr2=0x563be1572850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1572850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 -- 172.21.15.3:0/2075905785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 msgr2=0x563be1572d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be1572d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 -- 172.21.15.3:0/2075905785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563be12a9620 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da157a640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be1572d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1da0d79640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1572310 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d900df820 tx=0x7f1d900a52d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7be0ae9dc8f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.536+0000 7f1d7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2075905785 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90020070 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2075905785 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1d900a95b0 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d880316a0 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2075905785 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1d900dfc50 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 10 client.15519 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563be1583eb0 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2075905785 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d880316a0 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2075905785 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f1d900b1d20 con 0x563be1565e10 2024-05-08T05:55:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 10 client.15519 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 10 client.15519 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 20 client.15519 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be1585320 0x563be1587710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 4 client.15519 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563be14d91e0 con 0x563be1585320 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.537+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be1585320 0x563be1587710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.538+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be1585320 0x563be1587710 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce61b945aa1182c6 server_cookie=a5c3c13316762c5 in_seq=0 out_seq=0 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.538+0000 7f1d7bfff640 10 client.15519 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2075905785 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563be14d91e0 con 0x563be1585320 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1d7bfff640 10 client.15519 handle_command_reply: tid=0 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 client.15519 shutdown 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 msgr2=0x563be1572310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1572310 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d900df820 tx=0x7f1d900a52d0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 shutdown_connections 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be1585320 0x563be1587710 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be1572d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1572850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2075905785 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1572310 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1d7bfff640 0 client.15519 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 >> 172.21.15.3:0/2075905785 conn(0x563be1562420 msgr2=0x563be1568290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 shutdown_connections 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2075905785 wait complete. 2024-05-08T05:55:35.546 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.539+0000 7f1da5dd3740 20 client.15519 trim_cache size 0 max 16384 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.554+0000 7f1da5dd3740 1 Processor -- start 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.554+0000 7f1da5dd3740 1 -- start start 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1571470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1561400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be158cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1aba10 con 0x563be1565e10 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1ae2b0 con 0x563be1575500 2024-05-08T05:55:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1ac0a0 con 0x563be1566730 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1571470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1571470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36108/0 (socket says 172.21.15.3:36108) 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 -- 172.21.15.3:0/402392361 learned_addr learned my addr 172.21.15.3:0/402392361 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1561400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da157a640 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be158cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 -- 172.21.15.3:0/402392361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 msgr2=0x563be1561400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1561400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 -- 172.21.15.3:0/402392361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 msgr2=0x563be158cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be158cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 -- 172.21.15.3:0/402392361 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563be1566560 con 0x563be1565e10 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da157a640 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be158cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1da0d79640 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1571470 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000d740 tx=0x7f1d900a5aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969fa03539221603 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.555+0000 7f1d867fc640 1 -- 172.21.15.3:0/402392361 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d90020070 con 0x563be1565e10 2024-05-08T05:55:35.557 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.556+0000 7f1d867fc640 1 -- 172.21.15.3:0/402392361 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d900a9800 con 0x563be1565e10 2024-05-08T05:55:35.558 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.556+0000 7f1d867fc640 1 -- 172.21.15.3:0/402392361 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d900e2170 con 0x563be1565e10 2024-05-08T05:55:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.556+0000 7f1da5dd3740 1 -- 172.21.15.3:0/402392361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 msgr2=0x563be1571470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.556+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1571470 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000d740 tx=0x7f1d900a5aa0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 -- 172.21.15.3:0/402392361 shutdown_connections 2024-05-08T05:55:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be158cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.561 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1566730 0x563be1561400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/402392361 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1565e10 0x563be1571470 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 -- 172.21.15.3:0/402392361 >> 172.21.15.3:0/402392361 conn(0x563be1562420 msgr2=0x563be1560cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 -- 172.21.15.3:0/402392361 shutdown_connections 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 -- 172.21.15.3:0/402392361 wait complete. 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 -- start start 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.557+0000 7f1da5dd3740 1 Processor -- start 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1565e10 0x563be156f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 0x563be156f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be156fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1ac770 con 0x563be1566730 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1adbe0 con 0x563be1575500 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9c1ace40 con 0x563be1565e10 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 10 client.0 _ll_register_callbacks cb 0x7f1d9000de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 0x563be156f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da0d79640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1565e10 0x563be156f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 0x563be156f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:36122/0 (socket says 172.21.15.3:36122) 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 -- 172.21.15.3:0/2776670316 learned_addr learned my addr 172.21.15.3:0/2776670316 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da157a640 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be156fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 -- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1565e10 msgr2=0x563be156f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1565e10 0x563be156f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 -- 172.21.15.3:0/2776670316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 msgr2=0x563be156fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be156fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 -- 172.21.15.3:0/2776670316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563be157bc40 con 0x563be1566730 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d9bfff640 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 0x563be156f830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009930 tx=0x7f1d880050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29816e373ad0aec5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2776670316 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d88054070 con 0x563be1566730 2024-05-08T05:55:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2776670316 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1d880051d0 con 0x563be1566730 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.558+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d900a9800 con 0x563be1566730 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2776670316 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1d8805e050 con 0x563be1566730 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 10 client.15525 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563be15f0080 con 0x563be1566730 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2776670316 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d900a9800 con 0x563be1566730 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2776670316 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f1d8804d050 con 0x563be1566730 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 10 client.15525 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 20 client.15525 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be15f0410 0x563be15f2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 4 client.15525 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563be14d3050 con 0x563be15f0410 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.559+0000 7f1da0d79640 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be15f0410 0x563be15f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.560+0000 7f1da0d79640 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be15f0410 0x563be15f2800 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae40be868038a64f server_cookie=4a50916f35b729e9 in_seq=0 out_seq=0 2024-05-08T05:55:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.560+0000 7f1d7bfff640 10 client.15525 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1d7bfff640 1 -- 172.21.15.3:0/2776670316 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563be14d3050 con 0x563be15f0410 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1d7bfff640 10 client.15525 handle_command_reply: tid=0 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 client.15525 shutdown 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 msgr2=0x563be156f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 0x563be156f830 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009930 tx=0x7f1d880050c0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 shutdown_connections 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x563be15f0410 0x563be15f2800 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563be1575500 0x563be156fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x563be1566730 0x563be156f830 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 --2- 172.21.15.3:0/2776670316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563be1565e10 0x563be156f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 >> 172.21.15.3:0/2776670316 conn(0x563be1562420 msgr2=0x563be1573b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1d7bfff640 0 client.15525 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 shutdown_connections 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 1 -- 172.21.15.3:0/2776670316 wait complete. 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.564+0000 7f1da5dd3740 20 client.15525 trim_cache size 0 max 16384 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:55:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1d8c0b2950 msgr2=0x7f1d8c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1d8c0b2950 0x7f1d8c0b4e10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1d94005fd0 tx=0x7f1d94004a40 comp rx=0 tx=0).stop 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 msgr2=0x7f1d9c199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c199230 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1d9000d770 tx=0x7f1d9000dc30 comp rx=0 tx=0).stop 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 shutdown_connections 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1d8c0b2950 0x7f1d8c0b4e10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1d9c1078a0 0x7f1d9c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d9c102250 0x7f1d9c199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 --2- 172.21.15.3:0/1574256992 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1d9c101890 0x7f1d9c199230 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 >> 172.21.15.3:0/1574256992 conn(0x7f1d9c0789d0 msgr2=0x7f1d9c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 shutdown_connections 2024-05-08T05:55:35.569 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:35.565+0000 7f1da2fdd640 1 -- 172.21.15.3:0/1574256992 wait complete. 2024-05-08T05:55:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:35 smithi089 ceph-mon[34495]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-08T05:55:36.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:35 smithi104 ceph-mon[34559]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-08T05:55:36.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:35 smithi003 ceph-mon[30137]: pgmap v58: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-08T05:55:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:37 smithi089 ceph-mon[34495]: pgmap v59: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-08T05:55:38.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:37 smithi104 ceph-mon[34559]: pgmap v59: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-08T05:55:38.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:37 smithi003 ceph-mon[30137]: pgmap v59: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-08T05:55:38.349 INFO:tasks.workunit.client.0.smithi003.stderr:Updating files: 93% (11456/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T05:55:38.970 INFO:tasks.workunit.client.0.smithi003.stderr:Note: switching to 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'. 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T05:55:38.971 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr: git switch -c 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr:Or undo this operation with: 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr: git switch - 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:38.972 INFO:tasks.workunit.client.0.smithi003.stderr:HEAD is now at bee299f4ac3 mds: relax divergent backtrace scrub failures for replicated ancestor inodes 2024-05-08T05:55:38.977 DEBUG:teuthology.orchestra.run.smithi003:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T05:55:39.007 INFO:tasks.workunit.client.0.smithi003.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T05:55:39.009 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T05:55:39.009 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T05:55:39.712 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T05:55:39.770 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T05:55:39.812 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T05:55:39.813 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T05:55:39.813 INFO:tasks.workunit.client.0.smithi003.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T05:55:39.854 INFO:tasks.workunit.client.0.smithi003.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T05:55:39.857 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T05:55:39.857 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T05:55:39.915 INFO:tasks.workunit:Running workunits matching suites/fsync-tester.sh on client.0... 2024-05-08T05:55:39.916 INFO:tasks.workunit:Running workunit suites/fsync-tester.sh... 2024-05-08T05:55:39.916 DEBUG:teuthology.orchestra.run.smithi003:workunit test suites/fsync-tester.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-08T05:55:39.978 INFO:tasks.workunit.client.0.smithi003.stderr:+ wget http://download.ceph.com/qa/fsync-tester.c 2024-05-08T05:55:39.999 INFO:tasks.workunit.client.0.smithi003.stderr:--2024-05-08 05:55:39-- http://download.ceph.com/qa/fsync-tester.c 2024-05-08T05:55:40.000 INFO:tasks.workunit.client.0.smithi003.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-08T05:55:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:39 smithi089 ceph-mon[34495]: pgmap v60: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:39 smithi104 ceph-mon[34559]: pgmap v60: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:40.036 INFO:tasks.workunit.client.0.smithi003.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-08T05:55:40.072 INFO:tasks.workunit.client.0.smithi003.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-08T05:55:40.073 INFO:tasks.workunit.client.0.smithi003.stderr:Length: 3024 (3.0K) [application/octet-stream] 2024-05-08T05:55:40.073 INFO:tasks.workunit.client.0.smithi003.stderr:Saving to: ‘fsync-tester.c’ 2024-05-08T05:55:40.076 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:40.076 INFO:tasks.workunit.client.0.smithi003.stderr: 0K .. 100% 531M=0s 2024-05-08T05:55:40.076 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:40.076 INFO:tasks.workunit.client.0.smithi003.stderr:2024-05-08 05:55:40 (531 MB/s) - ‘fsync-tester.c’ saved [3024/3024] 2024-05-08T05:55:40.076 INFO:tasks.workunit.client.0.smithi003.stderr: 2024-05-08T05:55:40.076 INFO:tasks.workunit.client.0.smithi003.stderr:+ gcc -D_GNU_SOURCE fsync-tester.c -o fsync-tester 2024-05-08T05:55:40.143 INFO:tasks.workunit.client.0.smithi003.stderr:+ ./fsync-tester 2024-05-08T05:55:40.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:39 smithi003 ceph-mon[30137]: pgmap v60: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:41.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:41 smithi003 ceph-mon[30137]: pgmap v61: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:41 smithi089 ceph-mon[34495]: pgmap v61: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:41 smithi104 ceph-mon[34559]: pgmap v61: 129 pgs: 129 active+clean; 1.3 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T05:55:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:43 smithi104 ceph-mon[34559]: pgmap v62: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 264 KiB/s wr, 4 op/s 2024-05-08T05:55:44.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:43 smithi003 ceph-mon[30137]: pgmap v62: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 264 KiB/s wr, 4 op/s 2024-05-08T05:55:44.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:43 smithi089 ceph-mon[34495]: pgmap v62: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 264 KiB/s wr, 4 op/s 2024-05-08T05:55:46.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:45 smithi104 ceph-mon[34559]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 264 KiB/s wr, 3 op/s 2024-05-08T05:55:46.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:45 smithi003 ceph-mon[30137]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 264 KiB/s wr, 3 op/s 2024-05-08T05:55:46.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:45 smithi089 ceph-mon[34495]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 264 KiB/s wr, 3 op/s 2024-05-08T05:55:48.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:47 smithi104 ceph-mon[34559]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 264 KiB/s wr, 3 op/s 2024-05-08T05:55:48.065 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:47 smithi089 ceph-mon[34495]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 264 KiB/s wr, 3 op/s 2024-05-08T05:55:48.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:47 smithi003 ceph-mon[30137]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 264 KiB/s wr, 3 op/s 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 -- 172.21.15.3:0/2013609883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 msgr2=0x7f3978107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 --2- 172.21.15.3:0/2013609883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f3978107c80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3968009930 tx=0x7f396802a420 comp rx=0 tx=0).stop 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 -- 172.21.15.3:0/2013609883 shutdown_connections 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 --2- 172.21.15.3:0/2013609883 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3978102250 0x7f397810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 --2- 172.21.15.3:0/2013609883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 0x7f3978101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 --2- 172.21.15.3:0/2013609883 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f3978107c80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 -- 172.21.15.3:0/2013609883 >> 172.21.15.3:0/2013609883 conn(0x7f39780789d0 msgr2=0x7f3978078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 -- 172.21.15.3:0/2013609883 shutdown_connections 2024-05-08T05:55:48.662 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.661+0000 7f397c8da640 1 -- 172.21.15.3:0/2013609883 wait complete. 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.662+0000 7f397c8da640 1 Processor -- start 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 -- start start 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 0x7f39781980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3978102250 0x7f3978198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f397819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3978111b60 con 0x7f39781078a0 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39781119e0 con 0x7f3978102250 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397c8da640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3978111ce0 con 0x7f3978101890 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f397819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.665 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f397819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49360/0 (socket says 172.21.15.3:49360) 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 -- 172.21.15.3:0/3953181649 learned_addr learned my addr 172.21.15.3:0/3953181649 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397659c640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 0x7f39781980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3975d9b640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3978102250 0x7f3978198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 -- 172.21.15.3:0/3953181649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 msgr2=0x7f39781980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 0x7f39781980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 -- 172.21.15.3:0/3953181649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3978102250 msgr2=0x7f3978198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3978102250 0x7f3978198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f3976d9d640 1 -- 172.21.15.3:0/3953181649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397819d0d0 con 0x7f39781078a0 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.663+0000 7f397659c640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 0x7f39781980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.664+0000 7f3976d9d640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f397819c9a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f397000cce0 tx=0x7f397000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.664+0000 7f39677fe640 1 -- 172.21.15.3:0/3953181649 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397000f450 con 0x7f39781078a0 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.664+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397819d3c0 con 0x7f39781078a0 2024-05-08T05:55:48.666 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.664+0000 7f39677fe640 1 -- 172.21.15.3:0/3953181649 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f397000faa0 con 0x7f39781078a0 2024-05-08T05:55:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.664+0000 7f39677fe640 1 -- 172.21.15.3:0/3953181649 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970018b70 con 0x7f39781078a0 2024-05-08T05:55:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.664+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a4c50 con 0x7f39781078a0 2024-05-08T05:55:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.666+0000 7f39677fe640 1 -- 172.21.15.3:0/3953181649 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3970018d10 con 0x7f39781078a0 2024-05-08T05:55:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.667+0000 7f39677fe640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f39600b2950 0x7f39600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.667+0000 7f39677fe640 1 -- 172.21.15.3:0/3953181649 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f39700e2e40 con 0x7f39781078a0 2024-05-08T05:55:48.667 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.667+0000 7f397659c640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f39600b2950 0x7f39600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.668+0000 7f397659c640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f39600b2950 0x7f39600b4e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3968000c00 tx=0x7f3968009610 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.668+0000 7f397f5f6740 1 Processor -- start 2024-05-08T05:55:48.668 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.668+0000 7f397f5f6740 1 -- start start 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 0x55648f1412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f14bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f14c0f0 0x55648f14e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397f5f6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55648f0a65d0 con 0x55648f140f10 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397f5f6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55648f05f890 con 0x55648f14c0f0 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397f5f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55648f0a2590 con 0x55648f141830 2024-05-08T05:55:48.675 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 0x55648f1412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 0x55648f1412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49374/0 (socket says 172.21.15.3:49374) 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 -- 172.21.15.3:0/466594906 learned_addr learned my addr 172.21.15.3:0/466594906 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f3976d9d640 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f14c0f0 0x55648f14e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f3975d9b640 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f14bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 -- 172.21.15.3:0/466594906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 msgr2=0x55648f14bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f14bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 -- 172.21.15.3:0/466594906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f14c0f0 msgr2=0x55648f14e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f14c0f0 0x55648f14e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 -- 172.21.15.3:0/466594906 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648edb8210 con 0x55648f140f10 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f3975d9b640 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f397659c640 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 0x55648f1412f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3968009930 tx=0x7f396803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd38592707ffcbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f3964ff9640 1 -- 172.21.15.3:0/466594906 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968035050 con 0x55648f140f10 2024-05-08T05:55:48.676 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f3964ff9640 1 -- 172.21.15.3:0/466594906 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396804c070 con 0x55648f140f10 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.669+0000 7f3964ff9640 1 -- 172.21.15.3:0/466594906 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968004bc0 con 0x55648f140f10 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 -- 172.21.15.3:0/466594906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 msgr2=0x55648f1412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 0x55648f1412f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3968009930 tx=0x7f396803c000 comp rx=0 tx=0).stop 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 -- 172.21.15.3:0/466594906 shutdown_connections 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f14c0f0 0x55648f14e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f14bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 --2- 172.21.15.3:0/466594906 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f140f10 0x55648f1412f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 -- 172.21.15.3:0/466594906 >> 172.21.15.3:0/466594906 conn(0x55648f13d420 msgr2=0x55648f13d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 -- 172.21.15.3:0/466594906 shutdown_connections 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.670+0000 7f397f5f6740 1 -- 172.21.15.3:0/466594906 wait complete. 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 -- start start 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 Processor -- start 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f149040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.677 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f149580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55648f14ecc0 con 0x55648f14c0f0 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55648f063370 con 0x55648f141830 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55648f14ee40 con 0x55648f140f10 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397f5f6740 10 client.0 _ll_register_callbacks cb 0x55648edb7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49388/0 (socket says 172.21.15.3:49388) 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f3976d9d640 1 -- 172.21.15.3:0/1291645834 learned_addr learned my addr 172.21.15.3:0/1291645834 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f3975d9b640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.671+0000 7f397659c640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3976d9d640 1 -- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 msgr2=0x55648f149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3976d9d640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.678 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3976d9d640 1 -- 172.21.15.3:0/1291645834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 msgr2=0x55648f149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3976d9d640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f149580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3976d9d640 1 -- 172.21.15.3:0/1291645834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648ee84620 con 0x55648f14c0f0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3975d9b640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f397659c640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f3976d9d640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f149ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f39700a9ab0 tx=0x7f39700a95e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0f0b850f402b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f395dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f395dffb640 1 -- 172.21.15.3:0/1291645834 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700a9db0 con 0x55648f14c0f0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f395dffb640 1 -- 172.21.15.3:0/1291645834 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f397006f680 con 0x55648f14c0f0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f396804c070 con 0x55648f14c0f0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f395dffb640 1 -- 172.21.15.3:0/1291645834 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f39700ae160 con 0x55648f14c0f0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f397f5f6740 10 client.15537 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55648f15aab0 con 0x55648f14c0f0 2024-05-08T05:55:48.679 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f395dffb640 1 -- 172.21.15.3:0/1291645834 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396804c070 con 0x55648f14c0f0 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.672+0000 7f395dffb640 1 -- 172.21.15.3:0/1291645834 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f39700b2780 con 0x55648f14c0f0 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397f5f6740 10 client.15537 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397f5f6740 10 client.15537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397f5f6740 20 client.15537 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397f5f6740 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f15bf70 0x55648f15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397f5f6740 4 client.15537 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55648f0b41e0 con 0x55648f15bf70 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397659c640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f15bf70 0x55648f15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.680 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f395dffb640 10 client.15537 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.673+0000 7f397659c640 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f15bf70 0x55648f15e3a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d9ba6db9562eabf server_cookie=aff14562d2427948 in_seq=0 out_seq=0 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f395dffb640 1 -- 172.21.15.3:0/1291645834 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55648f0b41e0 con 0x55648f15bf70 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f395dffb640 10 client.15537 handle_command_reply: tid=0 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 client.15537 shutdown 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 msgr2=0x55648f149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f149ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f39700a9ab0 tx=0x7f39700a95e0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 shutdown_connections 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f15bf70 0x55648f15e3a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f149ac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 --2- 172.21.15.3:0/1291645834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 >> 172.21.15.3:0/1291645834 conn(0x55648f13d420 msgr2=0x55648f143150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:48.681 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f395dffb640 0 client.15537 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 shutdown_connections 2024-05-08T05:55:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.674+0000 7f397f5f6740 1 -- 172.21.15.3:0/1291645834 wait complete. 2024-05-08T05:55:48.682 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.675+0000 7f397f5f6740 20 client.15537 trim_cache size 0 max 16384 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 Processor -- start 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 -- start start 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f140f10 0x55648f1438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f1641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f39781acbd0 con 0x55648f14c0f0 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39781af470 con 0x55648f140f10 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397f5f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39781ad260 con 0x55648f141830 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f1641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.695 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f1641f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49396/0 (socket says 172.21.15.3:49396) 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f3976d9d640 1 -- 172.21.15.3:0/536406766 learned_addr learned my addr 172.21.15.3:0/536406766 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f3975d9b640 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.690+0000 7f397659c640 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f140f10 0x55648f1438d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f3976d9d640 1 -- 172.21.15.3:0/536406766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 msgr2=0x55648f163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f3976d9d640 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f3976d9d640 1 -- 172.21.15.3:0/536406766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f140f10 msgr2=0x55648f1438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f3976d9d640 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f140f10 0x55648f1438d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f3976d9d640 1 -- 172.21.15.3:0/536406766 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55648f143e10 con 0x55648f14c0f0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f397659c640 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f140f10 0x55648f1438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f3976d9d640 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f1641f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3970018650 tx=0x7f3970018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cede953bb2b7fa62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f395ffff640 1 -- 172.21.15.3:0/536406766 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700a9700 con 0x55648f14c0f0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f395ffff640 1 -- 172.21.15.3:0/536406766 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39700ad070 con 0x55648f14c0f0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.691+0000 7f395ffff640 1 -- 172.21.15.3:0/536406766 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397006f680 con 0x55648f14c0f0 2024-05-08T05:55:48.696 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 -- 172.21.15.3:0/536406766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 msgr2=0x55648f1641f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f1641f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3970018650 tx=0x7f3970018400 comp rx=0 tx=0).stop 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 -- 172.21.15.3:0/536406766 shutdown_connections 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f1641f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f141830 0x55648f163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 --2- 172.21.15.3:0/536406766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f140f10 0x55648f1438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 -- 172.21.15.3:0/536406766 >> 172.21.15.3:0/536406766 conn(0x55648f13d420 msgr2=0x55648f144330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 -- 172.21.15.3:0/536406766 shutdown_connections 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 -- 172.21.15.3:0/536406766 wait complete. 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 -- start start 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.692+0000 7f397f5f6740 1 Processor -- start 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f150620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f39781ad930 con 0x55648f14c0f0 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f39781aeda0 con 0x55648f141830 2024-05-08T05:55:48.697 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39781ae000 con 0x55648f140f10 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 10 client.0 _ll_register_callbacks cb 0x7f397000cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:49410/0 (socket says 172.21.15.3:49410) 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 -- 172.21.15.3:0/919459315 learned_addr learned my addr 172.21.15.3:0/919459315 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397659c640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3975d9b640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 -- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 msgr2=0x55648f14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 -- 172.21.15.3:0/919459315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 msgr2=0x55648f1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 -- 172.21.15.3:0/919459315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55648f1c5860 con 0x55648f14c0f0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f3976d9d640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f150620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f39700a7020 tx=0x7f39700a95e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cc4c0ee3fa0fb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f395dffb640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:55:48.698 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f395dffb640 1 -- 172.21.15.3:0/919459315 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39700a9700 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f395dffb640 1 -- 172.21.15.3:0/919459315 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f397000f800 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.693+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39700ad070 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f395dffb640 1 -- 172.21.15.3:0/919459315 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f39700b2c10 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 10 client.15543 fetch_fsmap learned FSMap version 23 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55648f1c6ab0 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f395dffb640 1 -- 172.21.15.3:0/919459315 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39700ad070 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f395dffb640 1 -- 172.21.15.3:0/919459315 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f39700af2f0 con 0x55648f14c0f0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 10 client.15543 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 20 client.15543 populate_metadata read hostname 'smithi003' 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f1c7160 0x55648f1c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 4 client.15543 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55648f0ae050 con 0x55648f1c7160 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.694+0000 7f397659c640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f1c7160 0x55648f1c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:55:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.695+0000 7f397659c640 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f1c7160 0x55648f1c9550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99efe60a421247fb server_cookie=7fb774e14bd8f8ef in_seq=0 out_seq=0 2024-05-08T05:55:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.695+0000 7f395dffb640 10 client.15543 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f395dffb640 1 -- 172.21.15.3:0/919459315 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55648f0ae050 con 0x55648f1c7160 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f395dffb640 10 client.15543 handle_command_reply: tid=0 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 client.15543 shutdown 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 msgr2=0x55648f150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f150620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f39700a7020 tx=0x7f39700a95e0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 shutdown_connections 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55648f1c7160 0x55648f1c9550 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.708 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55648f14c0f0 0x55648f150620 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55648f141830 0x55648f1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f395dffb640 0 client.15543 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 --2- 172.21.15.3:0/919459315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55648f140f10 0x55648f14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.707+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 >> 172.21.15.3:0/919459315 conn(0x55648f13d420 msgr2=0x55648f148fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 shutdown_connections 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397f5f6740 1 -- 172.21.15.3:0/919459315 wait complete. 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397f5f6740 20 client.15543 trim_cache size 0 max 16384 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:55:48.709 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:55:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:55:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f39600b2950 msgr2=0x7f39600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f39600b2950 0x7f39600b4e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3968000c00 tx=0x7f3968009610 comp rx=0 tx=0).stop 2024-05-08T05:55:48.710 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 msgr2=0x7f397819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f397819c9a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f397000cce0 tx=0x7f397000a620 comp rx=0 tx=0).stop 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 shutdown_connections 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f39600b2950 0x7f39600b4e10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f39781078a0 0x7f397819c9a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3978102250 0x7f3978198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 --2- 172.21.15.3:0/3953181649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3978101890 0x7f39781980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 >> 172.21.15.3:0/3953181649 conn(0x7f39780789d0 msgr2=0x7f3978104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 shutdown_connections 2024-05-08T05:55:48.711 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:55:48.708+0000 7f397c8da640 1 -- 172.21.15.3:0/3953181649 wait complete. 2024-05-08T05:55:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:49 smithi003 ceph-mon[30137]: pgmap v65: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 696 KiB/s wr, 5 op/s 2024-05-08T05:55:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:49 smithi089 ceph-mon[34495]: pgmap v65: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 696 KiB/s wr, 5 op/s 2024-05-08T05:55:50.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:49 smithi104 ceph-mon[34559]: pgmap v65: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 696 KiB/s wr, 5 op/s 2024-05-08T05:55:52.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:51 smithi003 ceph-mon[30137]: pgmap v66: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 695 KiB/s wr, 4 op/s 2024-05-08T05:55:52.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:51 smithi089 ceph-mon[34495]: pgmap v66: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 695 KiB/s wr, 4 op/s 2024-05-08T05:55:52.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:51 smithi104 ceph-mon[34559]: pgmap v66: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 695 KiB/s wr, 4 op/s 2024-05-08T05:55:54.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:53 smithi003 ceph-mon[30137]: pgmap v67: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-05-08T05:55:54.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:53 smithi089 ceph-mon[34495]: pgmap v67: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-05-08T05:55:54.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:53 smithi104 ceph-mon[34559]: pgmap v67: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.1 MiB/s wr, 10 op/s 2024-05-08T05:55:55.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:54 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:55:55.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:54 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:55:55.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:54 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:55:56.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:55 smithi003 ceph-mon[30137]: pgmap v68: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:55:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:55:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:55:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:55:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:55 smithi089 ceph-mon[34495]: pgmap v68: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:55:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:55:56.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:55:56.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:55:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:55 smithi104 ceph-mon[34559]: pgmap v68: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:55:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:55:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:55:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:55:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:57 smithi003 ceph-mon[30137]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:55:58.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:57 smithi089 ceph-mon[34495]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:55:58.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:57 smithi104 ceph-mon[34559]: pgmap v69: 129 pgs: 129 active+clean; 2.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:56:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:55:59 smithi003 ceph-mon[30137]: pgmap v70: 129 pgs: 129 active+clean; 2.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-05-08T05:56:00.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:55:59 smithi089 ceph-mon[34495]: pgmap v70: 129 pgs: 129 active+clean; 2.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-05-08T05:56:00.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:55:59 smithi104 ceph-mon[34559]: pgmap v70: 129 pgs: 129 active+clean; 2.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 9 op/s 2024-05-08T05:56:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 -- 172.21.15.3:0/2753334423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c1057f0 msgr2=0x7f7d6c109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 --2- 172.21.15.3:0/2753334423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c1057f0 0x7f7d6c109830 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d6002a410 comp rx=0 tx=0).stop 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 -- 172.21.15.3:0/2753334423 shutdown_connections 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 --2- 172.21.15.3:0/2753334423 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 0x7f7d6c111ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 --2- 172.21.15.3:0/2753334423 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c1057f0 0x7f7d6c109830 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 --2- 172.21.15.3:0/2753334423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c104e40 0x7f7d6c105220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 -- 172.21.15.3:0/2753334423 >> 172.21.15.3:0/2753334423 conn(0x7f7d6c100660 msgr2=0x7f7d6c102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 -- 172.21.15.3:0/2753334423 shutdown_connections 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.825+0000 7f7d710a0640 1 -- 172.21.15.3:0/2753334423 wait complete. 2024-05-08T05:56:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.826+0000 7f7d710a0640 1 Processor -- start 2024-05-08T05:56:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 -- start start 2024-05-08T05:56:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c104e40 0x7f7d6c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 0x7f7d6c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c19fd10 0x7f7d6c1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.829 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c114e70 con 0x7f7d6c104e40 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c114cf0 con 0x7f7d6c19fd10 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d710a0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c114ff0 con 0x7f7d6c109f10 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c104e40 0x7f7d6c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 0x7f7d6c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c104e40 0x7f7d6c19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41120/0 (socket says 172.21.15.3:41120) 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3199605389 learned_addr learned my addr 172.21.15.3:0/3199605389 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6b59e640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c19fd10 0x7f7d6c1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6b59e640 1 -- 172.21.15.3:0/3199605389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 msgr2=0x7f7d6c19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6b59e640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 0x7f7d6c19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6b59e640 1 -- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c104e40 msgr2=0x7f7d6c19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6b59e640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c104e40 0x7f7d6c19b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6b59e640 1 -- 172.21.15.3:0/3199605389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c1a0800 con 0x7f7d6c19fd10 2024-05-08T05:56:01.830 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.827+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 0x7f7d6c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.828+0000 7f7d6b59e640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c19fd10 0x7f7d6c1a01c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009fd0 tx=0x7f7d6400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.828+0000 7f7d53fff640 1 -- 172.21.15.3:0/3199605389 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400ce20 con 0x7f7d6c19fd10 2024-05-08T05:56:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.828+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c1a0af0 con 0x7f7d6c19fd10 2024-05-08T05:56:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.828+0000 7f7d53fff640 1 -- 172.21.15.3:0/3199605389 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d64005020 con 0x7f7d6c19fd10 2024-05-08T05:56:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.828+0000 7f7d53fff640 1 -- 172.21.15.3:0/3199605389 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64010690 con 0x7f7d6c19fd10 2024-05-08T05:56:01.831 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.828+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c1a8260 con 0x7f7d6c19fd10 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.831+0000 7f7d53fff640 1 -- 172.21.15.3:0/3199605389 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7d640054d0 con 0x7f7d6c19fd10 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.831+0000 7f7d73dc1740 1 Processor -- start 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 -- start start 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b022f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560e18a675d0 con 0x560e18b01f10 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560e18a20890 con 0x560e18b0d0f0 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e18a63590 con 0x560e18b02830 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.833 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b022f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41136/0 (socket says 172.21.15.3:41136) 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d6a59c640 1 -- 172.21.15.3:0/1138220886 learned_addr learned my addr 172.21.15.3:0/1138220886 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.832+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d53fff640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7d540b2950 0x7f7d540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6a59c640 1 -- 172.21.15.3:0/1138220886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 msgr2=0x560e18b0cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6a59c640 1 -- 172.21.15.3:0/1138220886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 msgr2=0x560e18b0f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.834 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6a59c640 1 -- 172.21.15.3:0/1138220886 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e18779210 con 0x560e18b01f10 2024-05-08T05:56:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d53fff640 1 -- 172.21.15.3:0/3199605389 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7d640e2fc0 con 0x7f7d6c19fd10 2024-05-08T05:56:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6b59e640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7d540b2950 0x7f7d540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.835 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b022f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7d60032fd0 tx=0x7f7d60005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774a8ffe233e9dad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d517fa640 1 -- 172.21.15.3:0/1138220886 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60035070 con 0x560e18b01f10 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d517fa640 1 -- 172.21.15.3:0/1138220886 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d600054d0 con 0x560e18b01f10 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.833+0000 7f7d517fa640 1 -- 172.21.15.3:0/1138220886 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60004af0 con 0x560e18b01f10 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 -- 172.21.15.3:0/1138220886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 msgr2=0x560e18b022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b022f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7d60032fd0 tx=0x7f7d60005b50 comp rx=0 tx=0).stop 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d6b59e640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7d540b2950 0x7f7d540b4e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000f80 tx=0x7f7d6409e7b0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.839 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 -- 172.21.15.3:0/1138220886 shutdown_connections 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/1138220886 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b022f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 -- 172.21.15.3:0/1138220886 >> 172.21.15.3:0/1138220886 conn(0x560e18afe420 msgr2=0x560e18afe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 -- 172.21.15.3:0/1138220886 shutdown_connections 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.834+0000 7f7d73dc1740 1 -- 172.21.15.3:0/1138220886 wait complete. 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 -- start start 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 Processor -- start 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b0a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.840 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x560e18b0fcc0 con 0x560e18b01f10 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560e18a24370 con 0x560e18b0d0f0 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e18b0fe40 con 0x560e18b02830 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d73dc1740 10 client.0 _ll_register_callbacks cb 0x560e18778a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b0a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b0a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41152/0 (socket says 172.21.15.3:41152) 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d6a59c640 1 -- 172.21.15.3:0/3830972416 learned_addr learned my addr 172.21.15.3:0/3830972416 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.835+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3830972416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 msgr2=0x560e18b0ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3830972416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 msgr2=0x560e18b0a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b0a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3830972416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e18845620 con 0x560e18b02830 2024-05-08T05:56:01.841 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b0a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0a5c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0127a0 tx=0x7f7d5c012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694e96de27ecea6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3830972416 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00f920 con 0x560e18b02830 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3830972416 <== mon.1 v2:172.21.15.104:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7d5c00fac0 con 0x560e18b02830 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.836+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d600054d0 con 0x560e18b02830 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3830972416 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7d5c025050 con 0x560e18b02830 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 10 client.25291 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560e18b1bbf0 con 0x560e18b02830 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3830972416 <== mon.1 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d600054d0 con 0x560e18b02830 2024-05-08T05:56:01.842 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3830972416 <== mon.1 v2:172.21.15.104:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7d5c0048c0 con 0x560e18b02830 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 10 client.25291 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 10 client.25291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 20 client.25291 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b1d0b0 0x560e18b1f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 4 client.25291 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e18a751e0 con 0x560e18b1d0b0 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.837+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b1d0b0 0x560e18b1f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d4eff5640 10 client.25291 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b1d0b0 0x560e18b1f4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89bb2b8c979c327 server_cookie=748821ac5896c124 in_seq=0 out_seq=0 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3830972416 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560e18a751e0 con 0x560e18b1d0b0 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d4eff5640 10 client.25291 handle_command_reply: tid=0 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 client.25291 shutdown 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 msgr2=0x560e18b0a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.843 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0a5c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0127a0 tx=0x7f7d5c012c60 comp rx=0 tx=0).stop 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 shutdown_connections 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b1d0b0 0x560e18b1f4a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b0ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b02830 0x560e18b0a5c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3830972416 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b01f10 0x560e18b0a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 >> 172.21.15.3:0/3830972416 conn(0x560e18afe420 msgr2=0x560e18b041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.838+0000 7f7d4eff5640 0 client.25291 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.839+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 shutdown_connections 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.839+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3830972416 wait complete. 2024-05-08T05:56:01.844 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.839+0000 7f7d73dc1740 20 client.25291 trim_cache size 0 max 16384 2024-05-08T05:56:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.858+0000 7f7d73dc1740 1 Processor -- start 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.858+0000 7f7d73dc1740 1 -- start start 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.858+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b01f10 0x560e18b048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b24db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1aff50 con 0x560e18b02830 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1b27f0 con 0x560e18b0d0f0 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1b05e0 con 0x560e18b01f10 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b24db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b01f10 0x560e18b048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b24db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41154/0 (socket says 172.21.15.3:41154) 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3483819161 learned_addr learned my addr 172.21.15.3:0/3483819161 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3483819161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b01f10 msgr2=0x560e18b048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b01f10 0x560e18b048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3483819161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 msgr2=0x560e18b252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b252f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3483819161 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e18afed60 con 0x560e18b02830 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.859+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b24db0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c000c00 tx=0x7f7d5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a032e10dd7e1a7ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d50ff9640 1 -- 172.21.15.3:0/3483819161 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c022040 con 0x560e18b02830 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d50ff9640 1 -- 172.21.15.3:0/3483819161 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c0122e0 con 0x560e18b02830 2024-05-08T05:56:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d50ff9640 1 -- 172.21.15.3:0/3483819161 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c005160 con 0x560e18b02830 2024-05-08T05:56:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3483819161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 msgr2=0x560e18b24db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b24db0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c000c00 tx=0x7f7d5c009f90 comp rx=0 tx=0).stop 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3483819161 shutdown_connections 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.860+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b0d0f0 0x560e18b252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b24db0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3483819161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b01f10 0x560e18b048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3483819161 >> 172.21.15.3:0/3483819161 conn(0x560e18afe420 msgr2=0x560e18b0c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3483819161 shutdown_connections 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3483819161 wait complete. 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- start start 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 Processor -- start 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b01f10 0x560e18b157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b10b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b0d0f0 0x560e18b11250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1b0cb0 con 0x560e18b02830 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1b2120 con 0x560e18b01f10 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c1b1380 con 0x560e18b0d0f0 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b10b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d6a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b01f10 0x560e18b157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d6ad9d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b10b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:41160/0 (socket says 172.21.15.3:41160) 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d6ad9d640 1 -- 172.21.15.3:0/3327249909 learned_addr learned my addr 172.21.15.3:0/3327249909 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.861+0000 7f7d73dc1740 10 client.0 _ll_register_callbacks cb 0x7f7d5c009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b0d0f0 0x560e18b11250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 -- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b0d0f0 msgr2=0x560e18b11250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b0d0f0 0x560e18b11250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 -- 172.21.15.3:0/3327249909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 msgr2=0x560e18b10b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:01.866 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b10b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 -- 172.21.15.3:0/3327249909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e18b86960 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d6a59c640 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b01f10 0x560e18b157a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d5c009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa216629c4f18032 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3327249909 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60035040 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c0122e0 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3327249909 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7d600584c0 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3327249909 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7d60031e20 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d73dc1740 10 client.25283 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.862+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x560e18b87b70 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3327249909 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d5c0122e0 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3327249909 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f7d60032680 con 0x560e18b01f10 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d73dc1740 10 client.25283 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:01.867 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d73dc1740 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d73dc1740 20 client.25283 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b87eb0 0x560e18b8a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d73dc1740 4 client.25283 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e18a6f050 con 0x560e18b87eb0 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.863+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b87eb0 0x560e18b8a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.864+0000 7f7d4eff5640 10 client.25283 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:01.868 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.864+0000 7f7d6ad9d640 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b87eb0 0x560e18b8a2d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5721b37b71dbc server_cookie=e9c9c8ea56279395 in_seq=0 out_seq=0 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d4eff5640 1 -- 172.21.15.3:0/3327249909 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e18a6f050 con 0x560e18b87eb0 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d4eff5640 10 client.25283 handle_command_reply: tid=0 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 client.25283 shutdown 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b01f10 msgr2=0x560e18b157a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b01f10 0x560e18b157a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d5c009f00 comp rx=0 tx=0).stop 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 shutdown_connections 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x560e18b87eb0 0x560e18b8a2d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.875 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e18b0d0f0 0x560e18b11250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x560e18b02830 0x560e18b10b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d4eff5640 0 client.25283 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 --2- 172.21.15.3:0/3327249909 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560e18b01f10 0x560e18b157a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.874+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 >> 172.21.15.3:0/3327249909 conn(0x560e18afe420 msgr2=0x560e18b0a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 shutdown_connections 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d73dc1740 1 -- 172.21.15.3:0/3327249909 wait complete. 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d73dc1740 20 client.25283 trim_cache size 0 max 16384 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:56:01.876 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:56:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7d540b2950 msgr2=0x7f7d540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d710a0640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7d540b2950 0x7f7d540b4e10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7d64000f80 tx=0x7f7d6409e7b0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c19fd10 msgr2=0x7f7d6c1a01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:01.877 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.875+0000 7f7d710a0640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c19fd10 0x7f7d6c1a01c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009fd0 tx=0x7f7d6400ef10 comp rx=0 tx=0).stop 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 shutdown_connections 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7d540b2950 0x7f7d540b4e10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7d6c19fd10 0x7f7d6c1a01c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d6c109f10 0x7f7d6c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 --2- 172.21.15.3:0/3199605389 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7d6c104e40 0x7f7d6c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 >> 172.21.15.3:0/3199605389 conn(0x7f7d6c100660 msgr2=0x7f7d6c1019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 shutdown_connections 2024-05-08T05:56:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:01.876+0000 7f7d710a0640 1 -- 172.21.15.3:0/3199605389 wait complete. 2024-05-08T05:56:02.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:01 smithi003 ceph-mon[30137]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:56:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:01 smithi089 ceph-mon[34495]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:56:02.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:01 smithi104 ceph-mon[34559]: pgmap v71: 129 pgs: 129 active+clean; 2.6 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 866 KiB/s wr, 7 op/s 2024-05-08T05:56:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:03 smithi003 ceph-mon[30137]: pgmap v72: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-05-08T05:56:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:03 smithi089 ceph-mon[34495]: pgmap v72: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-05-08T05:56:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:03 smithi104 ceph-mon[34559]: pgmap v72: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-05-08T05:56:06.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:05 smithi003 ceph-mon[30137]: pgmap v73: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-08T05:56:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:05 smithi089 ceph-mon[34495]: pgmap v73: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-08T05:56:06.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:05 smithi104 ceph-mon[34559]: pgmap v73: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-08T05:56:08.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:07 smithi003 ceph-mon[30137]: pgmap v74: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-08T05:56:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:07 smithi089 ceph-mon[34495]: pgmap v74: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-08T05:56:08.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:07 smithi104 ceph-mon[34559]: pgmap v74: 129 pgs: 129 active+clean; 239 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-05-08T05:56:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:09 smithi003 ceph-mon[30137]: pgmap v75: 129 pgs: 129 active+clean; 255 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-05-08T05:56:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:09 smithi089 ceph-mon[34495]: pgmap v75: 129 pgs: 129 active+clean; 255 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-05-08T05:56:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:09 smithi104 ceph-mon[34559]: pgmap v75: 129 pgs: 129 active+clean; 255 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 304 op/s 2024-05-08T05:56:12.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:11 smithi003 ceph-mon[30137]: pgmap v76: 129 pgs: 129 active+clean; 255 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-08T05:56:12.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:11 smithi089 ceph-mon[34495]: pgmap v76: 129 pgs: 129 active+clean; 255 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-08T05:56:12.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:11 smithi104 ceph-mon[34559]: pgmap v76: 129 pgs: 129 active+clean; 255 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.0 MiB/s wr, 302 op/s 2024-05-08T05:56:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:13 smithi003 ceph-mon[30137]: pgmap v77: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 788 op/s 2024-05-08T05:56:14.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:13 smithi089 ceph-mon[34495]: pgmap v77: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 788 op/s 2024-05-08T05:56:14.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:13 smithi104 ceph-mon[34559]: pgmap v77: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 18 MiB/s wr, 788 op/s 2024-05-08T05:56:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.991+0000 7f50fff78640 1 -- 172.21.15.3:0/2644493330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 msgr2=0x7f50f8101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.991+0000 7f50fff78640 1 --2- 172.21.15.3:0/2644493330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f8101d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009860 tx=0x7f50ec02a340 comp rx=0 tx=0).stop 2024-05-08T05:56:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 -- 172.21.15.3:0/2644493330 shutdown_connections 2024-05-08T05:56:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 --2- 172.21.15.3:0/2644493330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50f8102270 0x7f50f810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 --2- 172.21.15.3:0/2644493330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f8101d30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 --2- 172.21.15.3:0/2644493330 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f81078c0 0x7f50f8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 -- 172.21.15.3:0/2644493330 >> 172.21.15.3:0/2644493330 conn(0x7f50f8078a00 msgr2=0x7f50f8078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 -- 172.21.15.3:0/2644493330 shutdown_connections 2024-05-08T05:56:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 -- 172.21.15.3:0/2644493330 wait complete. 2024-05-08T05:56:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 Processor -- start 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.992+0000 7f50fff78640 1 -- start start 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fff78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fff78640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 0x7f50f819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fff78640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50f81078c0 0x7f50f81a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fff78640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111b40 con 0x7f50f8102270 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fff78640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81119c0 con 0x7f50f81018b0 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fff78640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111cc0 con 0x7f50f81078c0 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fd513640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 0x7f50f819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fd513640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 0x7f50f819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43028/0 (socket says 172.21.15.3:43028) 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fd513640 1 -- 172.21.15.3:0/2461135854 learned_addr learned my addr 172.21.15.3:0/2461135854 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fe515640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50f81078c0 0x7f50f81a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fdd14640 1 -- 172.21.15.3:0/2461135854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50f81078c0 msgr2=0x7f50f81a5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50f81078c0 0x7f50f81a5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fdd14640 1 -- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 msgr2=0x7f50f819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 0x7f50f819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fdd14640 1 -- 172.21.15.3:0/2461135854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f81a5650 con 0x7f50f81018b0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.993+0000 7f50fd513640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 0x7f50f819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.994+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f819a090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50f000ef60 tx=0x7f50f000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.994+0000 7f50eaffd640 1 -- 172.21.15.3:0/2461135854 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f001b070 con 0x7f50f81018b0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.994+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f81a5820 con 0x7f50f81018b0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.994+0000 7f50eaffd640 1 -- 172.21.15.3:0/2461135854 <== mon.2 v2:172.21.15.89:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f000ba20 con 0x7f50f81018b0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.994+0000 7f50eaffd640 1 -- 172.21.15.3:0/2461135854 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0016a70 con 0x7f50f81018b0 2024-05-08T05:56:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.994+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a5d90 con 0x7f50f81018b0 2024-05-08T05:56:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.997+0000 7f50eaffd640 1 -- 172.21.15.3:0/2461135854 <== mon.2 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f50f001e070 con 0x7f50f81018b0 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.997+0000 7f5102d69740 1 Processor -- start 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.997+0000 7f5102d69740 1 -- start start 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f5102d69740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a136700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f5102d69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f5102d69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f5102d69740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a0975d0 con 0x55dd9a136320 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f5102d69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a050890 con 0x55dd9a141500 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f5102d69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a093590 con 0x55dd9a136c40 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a136700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43036/0 (socket says 172.21.15.3:43036) 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 -- 172.21.15.3:0/1898390802 learned_addr learned my addr 172.21.15.3:0/1898390802 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50eaffd640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f50e40b2a50 0x7f50e40b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 -- 172.21.15.3:0/1898390802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 msgr2=0x55dd9a140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50eaffd640 1 -- 172.21.15.3:0/2461135854 <== mon.2 v2:172.21.15.89:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f50f00e2800 con 0x7f50f81018b0 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fdd14640 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 -- 172.21.15.3:0/1898390802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 msgr2=0x55dd9a1438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fd513640 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a1438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.998+0000 7f50fe515640 1 -- 172.21.15.3:0/1898390802 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd99da9210 con 0x55dd9a136320 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.999+0000 7f50fd513640 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.999+0000 7f50fe515640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f50e40b2a50 0x7f50e40b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.999+0000 7f50fe515640 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a136700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009d90 tx=0x7f50f4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12feb86555fd92fa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.999+0000 7f50e3fff640 1 -- 172.21.15.3:0/1898390802 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x55dd9a136320 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.999+0000 7f50e3fff640 1 -- 172.21.15.3:0/1898390802 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f40044b0 con 0x55dd9a136320 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:14.999+0000 7f50fe515640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f50e40b2a50 0x7f50e40b4f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50f4023000 tx=0x7f50f4024000 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f50e3fff640 1 -- 172.21.15.3:0/1898390802 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f400e6f0 con 0x55dd9a136320 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 -- 172.21.15.3:0/1898390802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 msgr2=0x55dd9a136700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a136700 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009d90 tx=0x7f50f4011810 comp rx=0 tx=0).stop 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 -- 172.21.15.3:0/1898390802 shutdown_connections 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 --2- 172.21.15.3:0/1898390802 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a136700 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.002 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.000+0000 7f5102d69740 1 -- 172.21.15.3:0/1898390802 >> 172.21.15.3:0/1898390802 conn(0x55dd9a132830 msgr2=0x55dd9a132c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.001+0000 7f5102d69740 1 -- 172.21.15.3:0/1898390802 shutdown_connections 2024-05-08T05:56:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.001+0000 7f5102d69740 1 -- 172.21.15.3:0/1898390802 wait complete. 2024-05-08T05:56:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.001+0000 7f5102d69740 1 -- start start 2024-05-08T05:56:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.001+0000 7f5102d69740 1 Processor -- start 2024-05-08T05:56:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a13e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.003 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a13ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a144180 con 0x55dd9a136320 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a054370 con 0x55dd9a141500 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dd9a144300 con 0x55dd9a136c40 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f50fdd14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a13e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f5102d69740 10 client.0 _ll_register_callbacks cb 0x55dd99da8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f50fdd14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a13e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43048/0 (socket says 172.21.15.3:43048) 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f50fdd14640 1 -- 172.21.15.3:0/2965334915 learned_addr learned my addr 172.21.15.3:0/2965334915 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f50fd513640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.002+0000 7f50fe515640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a13ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fdd14640 1 -- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 msgr2=0x55dd9a13e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fdd14640 1 -- 172.21.15.3:0/2965334915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 msgr2=0x55dd9a13ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a13ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fdd14640 1 -- 172.21.15.3:0/2965334915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd99e75620 con 0x55dd9a136320 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fd513640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50fdd14640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a13e3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f50f000cbd0 tx=0x7f50f00a9080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8240e80cdd83ca5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50e17fa640 1 -- 172.21.15.3:0/2965334915 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f00adae0 con 0x55dd9a136320 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50e17fa640 1 -- 172.21.15.3:0/2965334915 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f50f00e23f0 con 0x55dd9a136320 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f40044b0 con 0x55dd9a136320 2024-05-08T05:56:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f50e17fa640 1 -- 172.21.15.3:0/2965334915 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f50f00e2630 con 0x55dd9a136320 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f5102d69740 10 client.15573 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.003+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dd9a12dea0 con 0x55dd9a136320 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f50e17fa640 1 -- 172.21.15.3:0/2965334915 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f40044b0 con 0x55dd9a136320 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f50e17fa640 1 -- 172.21.15.3:0/2965334915 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f50f00e1030 con 0x55dd9a136320 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f5102d69740 10 client.15573 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f5102d69740 10 client.15573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f5102d69740 20 client.15573 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f5102d69740 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a12f770 0x55dd9a131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f5102d69740 4 client.15573 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd9a0a51e0 con 0x55dd9a12f770 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.004+0000 7f50fd513640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a12f770 0x55dd9a131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.005+0000 7f50fd513640 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a12f770 0x55dd9a131be0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb42f90fe6da725 server_cookie=a2e78de4148d6432 in_seq=0 out_seq=0 2024-05-08T05:56:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.005+0000 7f50e17fa640 10 client.15573 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.005+0000 7f50e17fa640 1 -- 172.21.15.3:0/2965334915 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dd9a0a51e0 con 0x55dd9a12f770 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.005+0000 7f50e17fa640 10 client.15573 handle_command_reply: tid=0 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.005+0000 7f5102d69740 1 client.15573 shutdown 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 msgr2=0x55dd9a13e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a13e3a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f50f000cbd0 tx=0x7f50f00a9080 comp rx=0 tx=0).stop 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 shutdown_connections 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a12f770 0x55dd9a131be0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a141500 0x55dd9a13ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f50e17fa640 0 client.15573 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 --2- 172.21.15.3:0/2965334915 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a13e3a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 >> 172.21.15.3:0/2965334915 conn(0x55dd9a132830 msgr2=0x55dd9a12d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 shutdown_connections 2024-05-08T05:56:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 1 -- 172.21.15.3:0/2965334915 wait complete. 2024-05-08T05:56:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.006+0000 7f5102d69740 20 client.15573 trim_cache size 0 max 16384 2024-05-08T05:56:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 Processor -- start 2024-05-08T05:56:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 -- start start 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136320 0x55dd9a13cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a141500 0x55dd9a155190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81abb50 con 0x55dd9a141500 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ae3f0 con 0x55dd9a136320 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f5102d69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ac1e0 con 0x55dd9a136c40 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f50fdd14640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136320 0x55dd9a13cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f50fe515640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a141500 0x55dd9a155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f50fdd14640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136320 0x55dd9a13cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:58494/0 (socket says 172.21.15.3:58494) 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f50fdd14640 1 -- 172.21.15.3:0/1730785983 learned_addr learned my addr 172.21.15.3:0/1730785983 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.028+0000 7f50fd513640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.030 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fd513640 1 -- 172.21.15.3:0/1730785983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136320 msgr2=0x55dd9a13cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fd513640 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136320 0x55dd9a13cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fd513640 1 -- 172.21.15.3:0/1730785983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a141500 msgr2=0x55dd9a155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fd513640 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a141500 0x55dd9a155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fd513640 1 -- 172.21.15.3:0/1730785983 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd9a132660 con 0x55dd9a136c40 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fe515640 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a141500 0x55dd9a155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50fd513640 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13d1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02a930 tx=0x7f50ec030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15257672c8716d96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.029+0000 7f50e37fe640 1 -- 172.21.15.3:0/1730785983 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec037040 con 0x55dd9a136c40 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.030+0000 7f50e37fe640 1 -- 172.21.15.3:0/1730785983 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ec030820 con 0x55dd9a136c40 2024-05-08T05:56:15.031 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.030+0000 7f50e37fe640 1 -- 172.21.15.3:0/1730785983 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0035f0 con 0x55dd9a136c40 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.030+0000 7f5102d69740 1 -- 172.21.15.3:0/1730785983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 msgr2=0x55dd9a13d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.030+0000 7f5102d69740 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13d1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f50ec02a930 tx=0x7f50ec030340 comp rx=0 tx=0).stop 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.030+0000 7f5102d69740 1 -- 172.21.15.3:0/1730785983 shutdown_connections 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a141500 0x55dd9a155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a136c40 0x55dd9a13d1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 --2- 172.21.15.3:0/1730785983 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136320 0x55dd9a13cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 -- 172.21.15.3:0/1730785983 >> 172.21.15.3:0/1730785983 conn(0x55dd9a132830 msgr2=0x55dd9a132c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 -- 172.21.15.3:0/1730785983 shutdown_connections 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 -- 172.21.15.3:0/1730785983 wait complete. 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 -- start start 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.031+0000 7f5102d69740 1 Processor -- start 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136c40 0x55dd9a149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a141500 0x55dd9a13a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81ac8b0 con 0x55dd9a136320 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81add20 con 0x55dd9a136c40 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81acf80 con 0x55dd9a141500 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f5102d69740 10 client.0 _ll_register_callbacks cb 0x7f50f0016410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fdd14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136c40 0x55dd9a149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fdd14640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a1495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:43074/0 (socket says 172.21.15.3:43074) 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fdd14640 1 -- 172.21.15.3:0/50903434 learned_addr learned my addr 172.21.15.3:0/50903434 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fe515640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a141500 0x55dd9a13a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 -- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a141500 msgr2=0x55dd9a13a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a141500 0x55dd9a13a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 -- 172.21.15.3:0/50903434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 msgr2=0x55dd9a1495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a1495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 -- 172.21.15.3:0/50903434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd9a13c3d0 con 0x55dd9a136c40 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50fd513640 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136c40 0x55dd9a149ae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009830 tx=0x7f50ec030f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9793904e53b7a02 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.032+0000 7f50e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.89:3300/0 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f50e17fa640 1 -- 172.21.15.3:0/50903434 <== mon.2 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec053070 con 0x55dd9a136c40 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f50e17fa640 1 -- 172.21.15.3:0/50903434 <== mon.2 v2:172.21.15.89:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f50ec004240 con 0x55dd9a136c40 2024-05-08T05:56:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ec030820 con 0x55dd9a136c40 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f50e17fa640 1 -- 172.21.15.3:0/50903434 <== mon.2 v2:172.21.15.89:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f50ec05f050 con 0x55dd9a136c40 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 10 client.25298 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55dd9a1b7350 con 0x55dd9a136c40 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f50e17fa640 1 -- 172.21.15.3:0/50903434 <== mon.2 v2:172.21.15.89:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50ec030820 con 0x55dd9a136c40 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f50e17fa640 1 -- 172.21.15.3:0/50903434 <== mon.2 v2:172.21.15.89:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f50ec0431d0 con 0x55dd9a136c40 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 10 client.25298 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 10 client.25298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 20 client.25298 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a1b7a80 0x55dd9a1b9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 4 client.25298 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.033+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd9a09f050 con 0x55dd9a1b7a80 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.034+0000 7f50fdd14640 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a1b7a80 0x55dd9a1b9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.034+0000 7f50e17fa640 10 client.25298 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.034+0000 7f50fdd14640 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a1b7a80 0x55dd9a1b9ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313a720b23414df9 server_cookie=158e971e328ca215 in_seq=0 out_seq=0 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f50e17fa640 1 -- 172.21.15.3:0/50903434 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd9a09f050 con 0x55dd9a1b7a80 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f50e17fa640 10 client.25298 handle_command_reply: tid=0 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 client.25298 shutdown 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136c40 msgr2=0x55dd9a149ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136c40 0x55dd9a149ae0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009830 tx=0x7f50ec030f50 comp rx=0 tx=0).stop 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 shutdown_connections 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55dd9a1b7a80 0x55dd9a1b9ea0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dd9a141500 0x55dd9a13a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55dd9a136c40 0x55dd9a149ae0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 --2- 172.21.15.3:0/50903434 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55dd9a136320 0x55dd9a1495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 >> 172.21.15.3:0/50903434 conn(0x55dd9a132830 msgr2=0x55dd9a13cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f50e17fa640 0 client.25298 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 shutdown_connections 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 1 -- 172.21.15.3:0/50903434 wait complete. 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.044+0000 7f5102d69740 20 client.25298 trim_cache size 0 max 16384 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:56:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:56:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f50e40b2a50 msgr2=0x7f50e40b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f50e40b2a50 0x7f50e40b4f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f50f4023000 tx=0x7f50f4024000 comp rx=0 tx=0).stop 2024-05-08T05:56:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 msgr2=0x7f50f819a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f819a090 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f50f000ef60 tx=0x7f50f000ef90 comp rx=0 tx=0).stop 2024-05-08T05:56:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 shutdown_connections 2024-05-08T05:56:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f50e40b2a50 0x7f50e40b4f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50f81078c0 0x7f50f81a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f50f8102270 0x7f50f819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 --2- 172.21.15.3:0/2461135854 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f50f81018b0 0x7f50f819a090 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 >> 172.21.15.3:0/2461135854 conn(0x7f50f8078a00 msgr2=0x7f50f8104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 shutdown_connections 2024-05-08T05:56:15.048 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:15.045+0000 7f50fff78640 1 -- 172.21.15.3:0/2461135854 wait complete. 2024-05-08T05:56:16.073 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:15 smithi104 ceph-mon[34559]: pgmap v78: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 691 op/s 2024-05-08T05:56:16.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:15 smithi003 ceph-mon[30137]: pgmap v78: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 691 op/s 2024-05-08T05:56:16.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:15 smithi089 ceph-mon[34495]: pgmap v78: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 691 op/s 2024-05-08T05:56:18.064 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:17 smithi089 ceph-mon[34495]: pgmap v79: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 691 op/s 2024-05-08T05:56:18.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:17 smithi003 ceph-mon[30137]: pgmap v79: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 691 op/s 2024-05-08T05:56:18.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:17 smithi104 ceph-mon[34559]: pgmap v79: 129 pgs: 129 active+clean; 259 MiB data, 490 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 17 MiB/s wr, 691 op/s 2024-05-08T05:56:20.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:19 smithi003 ceph-mon[30137]: pgmap v80: 129 pgs: 129 active+clean; 260 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 908 op/s 2024-05-08T05:56:20.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:19 smithi089 ceph-mon[34495]: pgmap v80: 129 pgs: 129 active+clean; 260 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 908 op/s 2024-05-08T05:56:20.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:19 smithi104 ceph-mon[34559]: pgmap v80: 129 pgs: 129 active+clean; 260 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 19 MiB/s wr, 908 op/s 2024-05-08T05:56:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:21 smithi003 ceph-mon[30137]: pgmap v81: 129 pgs: 129 active+clean; 260 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 702 op/s 2024-05-08T05:56:22.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:21 smithi089 ceph-mon[34495]: pgmap v81: 129 pgs: 129 active+clean; 260 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 702 op/s 2024-05-08T05:56:22.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:21 smithi104 ceph-mon[34559]: pgmap v81: 129 pgs: 129 active+clean; 260 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 18 MiB/s wr, 702 op/s 2024-05-08T05:56:24.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:23 smithi003 ceph-mon[30137]: pgmap v82: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 881 op/s 2024-05-08T05:56:24.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:23 smithi089 ceph-mon[34495]: pgmap v82: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 881 op/s 2024-05-08T05:56:24.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:23 smithi104 ceph-mon[34559]: pgmap v82: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 19 MiB/s wr, 881 op/s 2024-05-08T05:56:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:25 smithi003 ceph-mon[30137]: pgmap v83: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-05-08T05:56:26.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:25 smithi089 ceph-mon[34495]: pgmap v83: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-05-08T05:56:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:25 smithi104 ceph-mon[34559]: pgmap v83: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 -- 172.21.15.3:0/273423379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5200102250 msgr2=0x7f520010e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 --2- 172.21.15.3:0/273423379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5200102250 0x7f520010e760 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009930 tx=0x7f51f802a420 comp rx=0 tx=0).stop 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 -- 172.21.15.3:0/273423379 shutdown_connections 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 --2- 172.21.15.3:0/273423379 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5200102250 0x7f520010e760 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 --2- 172.21.15.3:0/273423379 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5200101890 0x7f5200101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 --2- 172.21.15.3:0/273423379 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52001078a0 0x7f5200107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 -- 172.21.15.3:0/273423379 >> 172.21.15.3:0/273423379 conn(0x7f52000789d0 msgr2=0x7f5200078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.140+0000 7f5206bc8640 1 -- 172.21.15.3:0/273423379 shutdown_connections 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 -- 172.21.15.3:0/273423379 wait complete. 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 Processor -- start 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 -- start start 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5200101890 0x7f52000fe850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5200102250 0x7f52000fed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 0x7f52000ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5200111b20 con 0x7f52001078a0 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52001119a0 con 0x7f5200101890 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.141+0000 7f5206bc8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5200111ca0 con 0x7f5200102250 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 0x7f52000ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 0x7f52000ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48038/0 (socket says 172.21.15.3:48038) 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 -- 172.21.15.3:0/3940704991 learned_addr learned my addr 172.21.15.3:0/3940704991 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5204964640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5200101890 0x7f52000fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f51fffff640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5200102250 0x7f52000fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 -- 172.21.15.3:0/3940704991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5200102250 msgr2=0x7f52000fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5200102250 0x7f52000fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 -- 172.21.15.3:0/3940704991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5200101890 msgr2=0x7f52000fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5200101890 0x7f52000fe850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 -- 172.21.15.3:0/3940704991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52001a76b0 con 0x7f52001078a0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5205165640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 0x7f52000ff2d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005940 tx=0x7f51f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f51fdffb640 1 -- 172.21.15.3:0/3940704991 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f803f030 con 0x7f52001078a0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001a7940 con 0x7f52001078a0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f51fdffb640 1 -- 172.21.15.3:0/3940704991 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f8003cb0 con 0x7f52001078a0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.142+0000 7f51fdffb640 1 -- 172.21.15.3:0/3940704991 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8035070 con 0x7f52001078a0 2024-05-08T05:56:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.143+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52001a7e30 con 0x7f52001078a0 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.145+0000 7f51fdffb640 1 -- 172.21.15.3:0/3940704991 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f51f8042030 con 0x7f52001078a0 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.145+0000 7f52099be740 1 Processor -- start 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 -- start start 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927e32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d5927485d0 con 0x55d5927e3830 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d592701890 con 0x55d5927e2f10 2024-05-08T05:56:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f52099be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d592744590 con 0x55d5927ee0f0 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927edbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48048/0 (socket says 172.21.15.3:48048) 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 -- 172.21.15.3:0/1367915866 learned_addr learned my addr 172.21.15.3:0/1367915866 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f51fffff640 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927e32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f51fffff640 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 -- 172.21.15.3:0/1367915866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 msgr2=0x55d5927f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 -- 172.21.15.3:0/1367915866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 msgr2=0x55d5927e32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927e32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f5204964640 1 -- 172.21.15.3:0/1367915866 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d59245a210 con 0x55d5927e3830 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f51fffff640 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.146+0000 7f51fdffb640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f51ec0b2950 0x7f51ec0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.147+0000 7f5204964640 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927edbb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bc50 tx=0x7f51e80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda5f4033dd9233c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.147+0000 7f51fffff640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f51ec0b2950 0x7f51ec0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.147+0000 7f51fdffb640 1 -- 172.21.15.3:0/3940704991 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f51f81031c0 con 0x7f52001078a0 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.147+0000 7f51f6ffd640 1 -- 172.21.15.3:0/1367915866 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e801c480 con 0x55d5927e3830 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.147+0000 7f51f6ffd640 1 -- 172.21.15.3:0/1367915866 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e801cad0 con 0x55d5927e3830 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.147+0000 7f51f6ffd640 1 -- 172.21.15.3:0/1367915866 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8011b00 con 0x55d5927e3830 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f51fffff640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f51ec0b2950 0x7f51ec0b4e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009fd0 tx=0x7f51f0014040 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 -- 172.21.15.3:0/1367915866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 msgr2=0x55d5927edbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927edbb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bc50 tx=0x7f51e80107a0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 -- 172.21.15.3:0/1367915866 shutdown_connections 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927edbb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 --2- 172.21.15.3:0/1367915866 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927e32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 -- 172.21.15.3:0/1367915866 >> 172.21.15.3:0/1367915866 conn(0x55d5927df420 msgr2=0x55d5927df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 -- 172.21.15.3:0/1367915866 shutdown_connections 2024-05-08T05:56:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 -- 172.21.15.3:0/1367915866 wait complete. 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 -- start start 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.148+0000 7f52099be740 1 Processor -- start 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927ebac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x55d5927f0d20 con 0x55d5927e3830 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d592705370 con 0x55d5927e2f10 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5927f0ea0 con 0x55d5927ee0f0 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f52099be740 10 client.0 _ll_register_callbacks cb 0x55d592459a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48062/0 (socket says 172.21.15.3:48062) 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 -- 172.21.15.3:0/233162350 learned_addr learned my addr 172.21.15.3:0/233162350 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f5204964640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f5205165640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927ebac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 -- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 msgr2=0x55d5927ebac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927ebac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 -- 172.21.15.3:0/233162350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 msgr2=0x55d5927eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 -- 172.21.15.3:0/233162350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d592526620 con 0x55d5927e3830 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51fffff640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927eb580 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005fd0 tx=0x7f51f0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12aa448f064d5322 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51dffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51dffff640 1 -- 172.21.15.3:0/233162350 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0018030 con 0x55d5927e3830 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.149+0000 7f51dffff640 1 -- 172.21.15.3:0/233162350 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f51f0003970 con 0x55d5927e3830 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51e801cad0 con 0x55d5927e3830 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f51dffff640 1 -- 172.21.15.3:0/233162350 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f51f0032050 con 0x55d5927e3830 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 10 client.15591 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d5927fcc40 con 0x55d5927e3830 2024-05-08T05:56:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f51dffff640 1 -- 172.21.15.3:0/233162350 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e801cad0 con 0x55d5927e3830 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f51dffff640 1 -- 172.21.15.3:0/233162350 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f51f000f040 con 0x55d5927e3830 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 10 client.15591 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 20 client.15591 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d5927fe100 0x55d592800570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 4 client.15591 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5927561e0 con 0x55d5927fe100 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.150+0000 7f5204964640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d5927fe100 0x55d592800570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.151+0000 7f51dffff640 10 client.15591 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.151+0000 7f5204964640 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d5927fe100 0x55d592800570 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f287e2ada452d1a server_cookie=8ad9cd9217297c2c in_seq=0 out_seq=0 2024-05-08T05:56:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.151+0000 7f51dffff640 1 -- 172.21.15.3:0/233162350 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d5927561e0 con 0x55d5927fe100 2024-05-08T05:56:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.151+0000 7f51dffff640 10 client.15591 handle_command_reply: tid=0 2024-05-08T05:56:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 client.15591 shutdown 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 msgr2=0x55d5927eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927eb580 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f51f0005fd0 tx=0x7f51f0005bd0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 shutdown_connections 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d5927fe100 0x55d592800570 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927ebac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f51dffff640 0 client.15591 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e3830 0x55d5927eb580 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 --2- 172.21.15.3:0/233162350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d5927eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 >> 172.21.15.3:0/233162350 conn(0x55d5927df420 msgr2=0x55d5927e51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 shutdown_connections 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 1 -- 172.21.15.3:0/233162350 wait complete. 2024-05-08T05:56:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.152+0000 7f52099be740 20 client.15591 trim_cache size 0 max 16384 2024-05-08T05:56:28.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:27 smithi003 ceph-mon[30137]: pgmap v84: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-05-08T05:56:28.170 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.168+0000 7f52099be740 1 Processor -- start 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.168+0000 7f52099be740 1 -- start start 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f52099be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d592805e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f52099be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927e3830 0x55d5928063c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f52099be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 0x55d5927ec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f52099be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52001aff10 con 0x55d5927ee0f0 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f52099be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52001b27b0 con 0x55d5927e2f10 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f52099be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52001b05a0 con 0x55d5927e3830 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 0x55d5927ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 0x55d5927ec870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48076/0 (socket says 172.21.15.3:48076) 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 -- 172.21.15.3:0/3854847420 learned_addr learned my addr 172.21.15.3:0/3854847420 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f51fffff640 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927e3830 0x55d5928063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5204964640 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d592805e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 -- 172.21.15.3:0/3854847420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927e3830 msgr2=0x55d5928063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927e3830 0x55d5928063c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 -- 172.21.15.3:0/3854847420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 msgr2=0x55d592805e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d592805e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 -- 172.21.15.3:0/3854847420 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5927e6be0 con 0x55d5927ee0f0 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5204964640 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d592805e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f5205165640 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 0x55d5927ec870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f51f8031440 tx=0x7f51f802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5041b1d2d0fdb8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f51f67fc640 1 -- 172.21.15.3:0/3854847420 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8049030 con 0x55d5927ee0f0 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.169+0000 7f51f67fc640 1 -- 172.21.15.3:0/3854847420 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f80ca1e0 con 0x55d5927ee0f0 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f51f67fc640 1 -- 172.21.15.3:0/3854847420 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f80c6600 con 0x55d5927ee0f0 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 -- 172.21.15.3:0/3854847420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 msgr2=0x55d5927ec870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 0x55d5927ec870 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f51f8031440 tx=0x7f51f802a420 comp rx=0 tx=0).stop 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 -- 172.21.15.3:0/3854847420 shutdown_connections 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927ee0f0 0x55d5927ec870 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927e3830 0x55d5928063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 --2- 172.21.15.3:0/3854847420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e2f10 0x55d592805e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 -- 172.21.15.3:0/3854847420 >> 172.21.15.3:0/3854847420 conn(0x55d5927df420 msgr2=0x55d5927df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 -- 172.21.15.3:0/3854847420 shutdown_connections 2024-05-08T05:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.170+0000 7f52099be740 1 -- 172.21.15.3:0/3854847420 wait complete. 2024-05-08T05:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 -- start start 2024-05-08T05:56:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 Processor -- start 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 0x55d592861d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e3830 0x55d592862280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f52001b0c70 con 0x55d5927e2f10 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f52001b20e0 con 0x55d5927e3830 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f52001b1340 con 0x55d5927ee0f0 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f52099be740 10 client.0 _ll_register_callbacks cb 0x7f51f0012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f5204964640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 0x55d592861d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f5204964640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 0x55d592861d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:48080/0 (socket says 172.21.15.3:48080) 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f5204964640 1 -- 172.21.15.3:0/3151889118 learned_addr learned my addr 172.21.15.3:0/3151889118 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.171+0000 7f51fffff640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e3830 0x55d592862280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5205165640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5204964640 1 -- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 msgr2=0x55d5927f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5204964640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5204964640 1 -- 172.21.15.3:0/3151889118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e3830 msgr2=0x55d592862280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5204964640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e3830 0x55d592862280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5204964640 1 -- 172.21.15.3:0/3151889118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5927f2b40 con 0x55d5927e2f10 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f5204964640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 0x55d592861d40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bc50 tx=0x7f51e8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f6c76f8b7ec9d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f51dffff640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f51dffff640 1 -- 172.21.15.3:0/3151889118 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8005220 con 0x55d5927e2f10 2024-05-08T05:56:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f80ca1e0 con 0x55d5927e2f10 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f51dffff640 1 -- 172.21.15.3:0/3151889118 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f51e80053c0 con 0x55d5927e2f10 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f51dffff640 1 -- 172.21.15.3:0/3151889118 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f51e80055c0 con 0x55d5927e2f10 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f52099be740 10 client.15597 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d592869470 con 0x55d5927e2f10 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.172+0000 7f51dffff640 1 -- 172.21.15.3:0/3151889118 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f80ca1e0 con 0x55d5927e2f10 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f51dffff640 1 -- 172.21.15.3:0/3151889118 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f51e80049b0 con 0x55d5927e2f10 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f52099be740 10 client.15597 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f52099be740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f52099be740 20 client.15597 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f52099be740 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d592869a80 0x55d59286bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f52099be740 4 client.15597 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:56:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d592750050 con 0x55d592869a80 2024-05-08T05:56:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f51fffff640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d592869a80 0x55d59286bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f51dffff640 10 client.15597 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:28.179 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.173+0000 7f51fffff640 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d592869a80 0x55d59286bea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97625e00988bd478 server_cookie=7788cecb0f5f6a89 in_seq=0 out_seq=0 2024-05-08T05:56:28.189 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f51dffff640 1 -- 172.21.15.3:0/3151889118 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d592750050 con 0x55d592869a80 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f51dffff640 10 client.15597 handle_command_reply: tid=0 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 client.15597 shutdown 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 msgr2=0x55d592861d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 0x55d592861d40 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f51e800bc50 tx=0x7f51e8005bd0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 shutdown_connections 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x55d592869a80 0x55d59286bea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5927ee0f0 0x55d5927f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d5927e3830 0x55d592862280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f51dffff640 0 client.15597 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:28.190 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 --2- 172.21.15.3:0/3151889118 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x55d5927e2f10 0x55d592861d40 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 >> 172.21.15.3:0/3151889118 conn(0x55d5927df420 msgr2=0x55d5927eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 shutdown_connections 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 1 -- 172.21.15.3:0/3151889118 wait complete. 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.189+0000 7f52099be740 20 client.15597 trim_cache size 0 max 16384 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:56:28.191 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:56:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f51ec0b2950 msgr2=0x7f51ec0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f51ec0b2950 0x7f51ec0b4e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009fd0 tx=0x7f51f0014040 comp rx=0 tx=0).stop 2024-05-08T05:56:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 msgr2=0x7f52000ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 0x7f52000ff2d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f51f8005940 tx=0x7f51f80042f0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.192 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 shutdown_connections 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f51ec0b2950 0x7f51ec0b4e10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f52001078a0 0x7f52000ff2d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5200102250 0x7f52000fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 --2- 172.21.15.3:0/3940704991 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5200101890 0x7f52000fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 >> 172.21.15.3:0/3940704991 conn(0x7f52000789d0 msgr2=0x7f5200100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 shutdown_connections 2024-05-08T05:56:28.193 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:28.190+0000 7f5206bc8640 1 -- 172.21.15.3:0/3940704991 wait complete. 2024-05-08T05:56:28.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:27 smithi089 ceph-mon[34495]: pgmap v84: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-05-08T05:56:28.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:27 smithi104 ceph-mon[34559]: pgmap v84: 129 pgs: 129 active+clean; 260 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-05-08T05:56:30.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:29 smithi003 ceph-mon[30137]: pgmap v85: 129 pgs: 129 active+clean; 372 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 2.08k op/s 2024-05-08T05:56:30.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:29 smithi089 ceph-mon[34495]: pgmap v85: 129 pgs: 129 active+clean; 372 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 2.08k op/s 2024-05-08T05:56:30.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:29 smithi104 ceph-mon[34559]: pgmap v85: 129 pgs: 129 active+clean; 372 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 2.08k op/s 2024-05-08T05:56:32.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:31 smithi003 ceph-mon[30137]: pgmap v86: 129 pgs: 129 active+clean; 372 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 1.86k op/s 2024-05-08T05:56:32.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:31 smithi089 ceph-mon[34495]: pgmap v86: 129 pgs: 129 active+clean; 372 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 1.86k op/s 2024-05-08T05:56:32.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:31 smithi104 ceph-mon[34559]: pgmap v86: 129 pgs: 129 active+clean; 372 MiB data, 741 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 1.86k op/s 2024-05-08T05:56:34.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:33 smithi003 ceph-mon[30137]: pgmap v87: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.87k op/s 2024-05-08T05:56:34.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:33 smithi089 ceph-mon[34495]: pgmap v87: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.87k op/s 2024-05-08T05:56:34.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:33 smithi104 ceph-mon[34559]: pgmap v87: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 15 MiB/s wr, 1.87k op/s 2024-05-08T05:56:36.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:35 smithi003 ceph-mon[30137]: pgmap v88: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.69k op/s 2024-05-08T05:56:36.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:35 smithi089 ceph-mon[34495]: pgmap v88: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.69k op/s 2024-05-08T05:56:36.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:35 smithi104 ceph-mon[34559]: pgmap v88: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.69k op/s 2024-05-08T05:56:38.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:37 smithi003 ceph-mon[30137]: pgmap v89: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.69k op/s 2024-05-08T05:56:38.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:37 smithi089 ceph-mon[34495]: pgmap v89: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.69k op/s 2024-05-08T05:56:38.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:37 smithi104 ceph-mon[34559]: pgmap v89: 129 pgs: 129 active+clean; 381 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 1.69k op/s 2024-05-08T05:56:40.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:39 smithi003 ceph-mon[30137]: pgmap v90: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2024-05-08T05:56:40.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:39 smithi089 ceph-mon[34495]: pgmap v90: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2024-05-08T05:56:40.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:39 smithi104 ceph-mon[34559]: pgmap v90: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 30 MiB/s wr, 1.70k op/s 2024-05-08T05:56:40.714 INFO:tasks.workunit.client.0.smithi003.stdout:setting up random write file 2024-05-08T05:56:40.714 INFO:tasks.workunit.client.0.smithi003.stdout:done setting up random write file 2024-05-08T05:56:40.714 INFO:tasks.workunit.client.0.smithi003.stdout:starting fsync run 2024-05-08T05:56:40.714 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0079s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0050s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0048s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0046s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0043s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0044s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0051s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0047s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0048s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0047s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0045s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0044s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0044s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0048s 2024-05-08T05:56:40.715 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0046s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0042s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0044s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0045s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0045s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0055s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0043s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0037s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0043s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0042s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0038s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0042s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0037s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0038s 2024-05-08T05:56:40.716 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0044s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0046s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0042s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0040s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0040s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0041s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0039s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0036s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0042s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0042s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0040s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0042s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0041s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0039s 2024-05-08T05:56:40.717 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0005s fsync time: 0.0041s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0041s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.1381s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0048s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0044s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0046s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0048s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0051s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0044s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0043s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0047s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0046s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0049s 2024-05-08T05:56:40.718 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0049s 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0050s 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0003s fsync time: 0.0049s 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0047s 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stdout:write time: 0.0002s fsync time: 0.0047s 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stdout:run done 61 fsyncs total, killing random writer 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stderr:+ echo /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-08T05:56:40.719 INFO:tasks.workunit.client.0.smithi003.stdout:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2024-05-08T05:56:40.720 INFO:tasks.workunit.client.0.smithi003.stderr:+ whereis lsof 2024-05-08T05:56:40.758 INFO:tasks.workunit.client.0.smithi003.stdout:lsof: /usr/bin/lsof /usr/share/man/man1/lsof.1.gz 2024-05-08T05:56:40.758 INFO:tasks.workunit.client.0.smithi003.stderr:+ lsof 2024-05-08T05:56:40.776 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/57ee78c12b223261b9c82c4ee8eede80dc882cb7cb669ce87249f012366e4608/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/d11e0200615e4bf9cd0647f78679d037705ac7825f7ba0a12ab7a28a5babfb8d/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/428e12a219ca223073d3de86e1bff0b4e822d70f54dcbf79bd1f3873a21e452c/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/1e51fb652e6a9bde266293d2f6123cbcd81fac037890469cbaae0bafe668b5ea/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/566c54d2a453dde747a2b1685a971f01a3c9666352c75351202f5659ec8256bb/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/42f0b37bf7d756d21af522534fbecc2663a614f5952760bc97436760e12a60b7/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/6a94c2d7b24c50c84e9f925414cc84d402dc5b95373b791c85863bbdae5f0c43/merged 2024-05-08T05:56:40.777 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.778 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/8a7831d33e4791524adcabcfc70eb064bda792fb972420abb4afb6f255b15642/merged 2024-05-08T05:56:40.778 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.778 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/24144dbabd23c9eeb7721c419cd5923a5416ce6e06dfd930cd15d71e18ad094e/merged 2024-05-08T05:56:40.778 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.778 INFO:tasks.workunit.client.0.smithi003.stderr:lsof: WARNING: can't stat() overlay file system /var/lib/containers/storage/overlay/fa8fd93e6e43de069e25add78552362673b857890db7a1a63f323cd6026216c6/merged 2024-05-08T05:56:40.778 INFO:tasks.workunit.client.0.smithi003.stderr: Output information may be incomplete. 2024-05-08T05:56:40.835 INFO:tasks.workunit.client.0.smithi003.stdout:COMMAND PID TID TASKCMD USER FD TYPE DEVICE SIZE/OFF NODE NAME 2024-05-08T05:56:40.835 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 1 root cwd unknown /proc/1/cwd (readlink: Permission denied) 2024-05-08T05:56:40.835 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 1 root rtd unknown /proc/1/root (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 1 root txt unknown /proc/1/exe (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 1 root NOFD /proc/1/fd (opendir: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:kthreadd 2 root cwd unknown /proc/2/cwd (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:kthreadd 2 root rtd unknown /proc/2/root (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:kthreadd 2 root txt unknown /proc/2/exe (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:kthreadd 2 root NOFD /proc/2/fd (opendir: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_gp 3 root cwd unknown /proc/3/cwd (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_gp 3 root rtd unknown /proc/3/root (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_gp 3 root txt unknown /proc/3/exe (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_gp 3 root NOFD /proc/3/fd (opendir: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_par_g 4 root cwd unknown /proc/4/cwd (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_par_g 4 root rtd unknown /proc/4/root (readlink: Permission denied) 2024-05-08T05:56:40.836 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_par_g 4 root txt unknown /proc/4/exe (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_par_g 4 root NOFD /proc/4/fd (opendir: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:slub_flus 5 root cwd unknown /proc/5/cwd (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:slub_flus 5 root rtd unknown /proc/5/root (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:slub_flus 5 root txt unknown /proc/5/exe (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:slub_flus 5 root NOFD /proc/5/fd (opendir: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:netns 6 root cwd unknown /proc/6/cwd (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:netns 6 root rtd unknown /proc/6/root (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:netns 6 root txt unknown /proc/6/exe (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:netns 6 root NOFD /proc/6/fd (opendir: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 8 root cwd unknown /proc/8/cwd (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 8 root rtd unknown /proc/8/root (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 8 root txt unknown /proc/8/exe (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 8 root NOFD /proc/8/fd (opendir: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:mm_percpu 10 root cwd unknown /proc/10/cwd (readlink: Permission denied) 2024-05-08T05:56:40.837 INFO:tasks.workunit.client.0.smithi003.stdout:mm_percpu 10 root rtd unknown /proc/10/root (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:mm_percpu 10 root txt unknown /proc/10/exe (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:mm_percpu 10 root NOFD /proc/10/fd (opendir: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 11 root cwd unknown /proc/11/cwd (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 11 root rtd unknown /proc/11/root (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 11 root txt unknown /proc/11/exe (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 11 root NOFD /proc/11/fd (opendir: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 12 root cwd unknown /proc/12/cwd (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 12 root rtd unknown /proc/12/root (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 12 root txt unknown /proc/12/exe (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 12 root NOFD /proc/12/fd (opendir: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 13 root cwd unknown /proc/13/cwd (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 13 root rtd unknown /proc/13/root (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 13 root txt unknown /proc/13/exe (readlink: Permission denied) 2024-05-08T05:56:40.838 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 13 root NOFD /proc/13/fd (opendir: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 14 root cwd unknown /proc/14/cwd (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 14 root rtd unknown /proc/14/root (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 14 root txt unknown /proc/14/exe (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_tasks 14 root NOFD /proc/14/fd (opendir: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 15 root cwd unknown /proc/15/cwd (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 15 root rtd unknown /proc/15/root (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 15 root txt unknown /proc/15/exe (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 15 root NOFD /proc/15/fd (opendir: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:pr/ttyS1 16 root cwd unknown /proc/16/cwd (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:pr/ttyS1 16 root rtd unknown /proc/16/root (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:pr/ttyS1 16 root txt unknown /proc/16/exe (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:pr/ttyS1 16 root NOFD /proc/16/fd (opendir: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_preem 17 root cwd unknown /proc/17/cwd (readlink: Permission denied) 2024-05-08T05:56:40.839 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_preem 17 root rtd unknown /proc/17/root (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_preem 17 root txt unknown /proc/17/exe (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:rcu_preem 17 root NOFD /proc/17/fd (opendir: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:migration 18 root cwd unknown /proc/18/cwd (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:migration 18 root rtd unknown /proc/18/root (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:migration 18 root txt unknown /proc/18/exe (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:migration 18 root NOFD /proc/18/fd (opendir: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 19 root cwd unknown /proc/19/cwd (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 19 root rtd unknown /proc/19/root (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 19 root txt unknown /proc/19/exe (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 19 root NOFD /proc/19/fd (opendir: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/0 21 root cwd unknown /proc/21/cwd (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/0 21 root rtd unknown /proc/21/root (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/0 21 root txt unknown /proc/21/exe (readlink: Permission denied) 2024-05-08T05:56:40.840 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/0 21 root NOFD /proc/21/fd (opendir: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/1 22 root cwd unknown /proc/22/cwd (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/1 22 root rtd unknown /proc/22/root (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/1 22 root txt unknown /proc/22/exe (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/1 22 root NOFD /proc/22/fd (opendir: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 23 root cwd unknown /proc/23/cwd (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 23 root rtd unknown /proc/23/root (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 23 root txt unknown /proc/23/exe (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 23 root NOFD /proc/23/fd (opendir: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:migration 24 root cwd unknown /proc/24/cwd (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:migration 24 root rtd unknown /proc/24/root (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:migration 24 root txt unknown /proc/24/exe (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:migration 24 root NOFD /proc/24/fd (opendir: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 25 root cwd unknown /proc/25/cwd (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 25 root rtd unknown /proc/25/root (readlink: Permission denied) 2024-05-08T05:56:40.841 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 25 root txt unknown /proc/25/exe (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 25 root NOFD /proc/25/fd (opendir: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 27 root cwd unknown /proc/27/cwd (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 27 root rtd unknown /proc/27/root (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 27 root txt unknown /proc/27/exe (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 27 root NOFD /proc/27/fd (opendir: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/2 28 root cwd unknown /proc/28/cwd (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/2 28 root rtd unknown /proc/28/root (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/2 28 root txt unknown /proc/28/exe (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/2 28 root NOFD /proc/28/fd (opendir: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 29 root cwd unknown /proc/29/cwd (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 29 root rtd unknown /proc/29/root (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 29 root txt unknown /proc/29/exe (readlink: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 29 root NOFD /proc/29/fd (opendir: Permission denied) 2024-05-08T05:56:40.842 INFO:tasks.workunit.client.0.smithi003.stdout:migration 30 root cwd unknown /proc/30/cwd (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:migration 30 root rtd unknown /proc/30/root (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:migration 30 root txt unknown /proc/30/exe (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:migration 30 root NOFD /proc/30/fd (opendir: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 31 root cwd unknown /proc/31/cwd (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 31 root rtd unknown /proc/31/root (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 31 root txt unknown /proc/31/exe (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 31 root NOFD /proc/31/fd (opendir: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 33 root cwd unknown /proc/33/cwd (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 33 root rtd unknown /proc/33/root (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 33 root txt unknown /proc/33/exe (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 33 root NOFD /proc/33/fd (opendir: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/3 34 root cwd unknown /proc/34/cwd (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/3 34 root rtd unknown /proc/34/root (readlink: Permission denied) 2024-05-08T05:56:40.843 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/3 34 root txt unknown /proc/34/exe (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/3 34 root NOFD /proc/34/fd (opendir: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 35 root cwd unknown /proc/35/cwd (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 35 root rtd unknown /proc/35/root (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 35 root txt unknown /proc/35/exe (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 35 root NOFD /proc/35/fd (opendir: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:migration 36 root cwd unknown /proc/36/cwd (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:migration 36 root rtd unknown /proc/36/root (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:migration 36 root txt unknown /proc/36/exe (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:migration 36 root NOFD /proc/36/fd (opendir: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 37 root cwd unknown /proc/37/cwd (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 37 root rtd unknown /proc/37/root (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 37 root txt unknown /proc/37/exe (readlink: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 37 root NOFD /proc/37/fd (opendir: Permission denied) 2024-05-08T05:56:40.844 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 39 root cwd unknown /proc/39/cwd (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 39 root rtd unknown /proc/39/root (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 39 root txt unknown /proc/39/exe (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 39 root NOFD /proc/39/fd (opendir: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/4 40 root cwd unknown /proc/40/cwd (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/4 40 root rtd unknown /proc/40/root (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/4 40 root txt unknown /proc/40/exe (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/4 40 root NOFD /proc/40/fd (opendir: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 41 root cwd unknown /proc/41/cwd (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 41 root rtd unknown /proc/41/root (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 41 root txt unknown /proc/41/exe (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 41 root NOFD /proc/41/fd (opendir: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:migration 42 root cwd unknown /proc/42/cwd (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:migration 42 root rtd unknown /proc/42/root (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:migration 42 root txt unknown /proc/42/exe (readlink: Permission denied) 2024-05-08T05:56:40.845 INFO:tasks.workunit.client.0.smithi003.stdout:migration 42 root NOFD /proc/42/fd (opendir: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 43 root cwd unknown /proc/43/cwd (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 43 root rtd unknown /proc/43/root (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 43 root txt unknown /proc/43/exe (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 43 root NOFD /proc/43/fd (opendir: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 44 root cwd unknown /proc/44/cwd (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 44 root rtd unknown /proc/44/root (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 44 root txt unknown /proc/44/exe (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 44 root NOFD /proc/44/fd (opendir: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 45 root cwd unknown /proc/45/cwd (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 45 root rtd unknown /proc/45/root (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 45 root txt unknown /proc/45/exe (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 45 root NOFD /proc/45/fd (opendir: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/5 46 root cwd unknown /proc/46/cwd (readlink: Permission denied) 2024-05-08T05:56:40.846 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/5 46 root rtd unknown /proc/46/root (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/5 46 root txt unknown /proc/46/exe (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/5 46 root NOFD /proc/46/fd (opendir: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 47 root cwd unknown /proc/47/cwd (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 47 root rtd unknown /proc/47/root (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 47 root txt unknown /proc/47/exe (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 47 root NOFD /proc/47/fd (opendir: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:migration 48 root cwd unknown /proc/48/cwd (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:migration 48 root rtd unknown /proc/48/root (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:migration 48 root txt unknown /proc/48/exe (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:migration 48 root NOFD /proc/48/fd (opendir: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 49 root cwd unknown /proc/49/cwd (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 49 root rtd unknown /proc/49/root (readlink: Permission denied) 2024-05-08T05:56:40.847 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 49 root txt unknown /proc/49/exe (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 49 root NOFD /proc/49/fd (opendir: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 51 root cwd unknown /proc/51/cwd (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 51 root rtd unknown /proc/51/root (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 51 root txt unknown /proc/51/exe (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 51 root NOFD /proc/51/fd (opendir: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/6 52 root cwd unknown /proc/52/cwd (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/6 52 root rtd unknown /proc/52/root (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/6 52 root txt unknown /proc/52/exe (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/6 52 root NOFD /proc/52/fd (opendir: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 53 root cwd unknown /proc/53/cwd (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 53 root rtd unknown /proc/53/root (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 53 root txt unknown /proc/53/exe (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 53 root NOFD /proc/53/fd (opendir: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:migration 54 root cwd unknown /proc/54/cwd (readlink: Permission denied) 2024-05-08T05:56:40.848 INFO:tasks.workunit.client.0.smithi003.stdout:migration 54 root rtd unknown /proc/54/root (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:migration 54 root txt unknown /proc/54/exe (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:migration 54 root NOFD /proc/54/fd (opendir: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 55 root cwd unknown /proc/55/cwd (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 55 root rtd unknown /proc/55/root (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 55 root txt unknown /proc/55/exe (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 55 root NOFD /proc/55/fd (opendir: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 57 root cwd unknown /proc/57/cwd (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 57 root rtd unknown /proc/57/root (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 57 root txt unknown /proc/57/exe (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 57 root NOFD /proc/57/fd (opendir: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/7 58 root cwd unknown /proc/58/cwd (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/7 58 root rtd unknown /proc/58/root (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/7 58 root txt unknown /proc/58/exe (readlink: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:cpuhp/7 58 root NOFD /proc/58/fd (opendir: Permission denied) 2024-05-08T05:56:40.849 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 59 root cwd unknown /proc/59/cwd (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 59 root rtd unknown /proc/59/root (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 59 root txt unknown /proc/59/exe (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:idle_inje 59 root NOFD /proc/59/fd (opendir: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:migration 60 root cwd unknown /proc/60/cwd (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:migration 60 root rtd unknown /proc/60/root (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:migration 60 root txt unknown /proc/60/exe (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:migration 60 root NOFD /proc/60/fd (opendir: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 61 root cwd unknown /proc/61/cwd (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 61 root rtd unknown /proc/61/root (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 61 root txt unknown /proc/61/exe (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:ksoftirqd 61 root NOFD /proc/61/fd (opendir: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 63 root cwd unknown /proc/63/cwd (readlink: Permission denied) 2024-05-08T05:56:40.850 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 63 root rtd unknown /proc/63/root (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 63 root txt unknown /proc/63/exe (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 63 root NOFD /proc/63/fd (opendir: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 65 root cwd unknown /proc/65/cwd (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 65 root rtd unknown /proc/65/root (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 65 root txt unknown /proc/65/exe (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 65 root NOFD /proc/65/fd (opendir: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69 root cwd unknown /proc/69/cwd (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69 root rtd unknown /proc/69/root (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69 root txt unknown /proc/69/exe (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69 root NOFD /proc/69/fd (opendir: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 70 root cwd unknown /proc/70/cwd (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 70 root rtd unknown /proc/70/root (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 70 root txt unknown /proc/70/exe (readlink: Permission denied) 2024-05-08T05:56:40.851 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 70 root NOFD /proc/70/fd (opendir: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kdevtmpfs 71 root cwd unknown /proc/71/cwd (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kdevtmpfs 71 root rtd unknown /proc/71/root (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kdevtmpfs 71 root txt unknown /proc/71/exe (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kdevtmpfs 71 root NOFD /proc/71/fd (opendir: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:inet_frag 72 root cwd unknown /proc/72/cwd (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:inet_frag 72 root rtd unknown /proc/72/root (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:inet_frag 72 root txt unknown /proc/72/exe (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:inet_frag 72 root NOFD /proc/72/fd (opendir: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kauditd 73 root cwd unknown /proc/73/cwd (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kauditd 73 root rtd unknown /proc/73/root (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kauditd 73 root txt unknown /proc/73/exe (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:kauditd 73 root NOFD /proc/73/fd (opendir: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:khungtask 75 root cwd unknown /proc/75/cwd (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:khungtask 75 root rtd unknown /proc/75/root (readlink: Permission denied) 2024-05-08T05:56:40.852 INFO:tasks.workunit.client.0.smithi003.stdout:khungtask 75 root txt unknown /proc/75/exe (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:khungtask 75 root NOFD /proc/75/fd (opendir: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:oom_reape 76 root cwd unknown /proc/76/cwd (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:oom_reape 76 root rtd unknown /proc/76/root (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:oom_reape 76 root txt unknown /proc/76/exe (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:oom_reape 76 root NOFD /proc/76/fd (opendir: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:writeback 77 root cwd unknown /proc/77/cwd (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:writeback 77 root rtd unknown /proc/77/root (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:writeback 77 root txt unknown /proc/77/exe (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:writeback 77 root NOFD /proc/77/fd (opendir: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:kcompactd 78 root cwd unknown /proc/78/cwd (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:kcompactd 78 root rtd unknown /proc/78/root (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:kcompactd 78 root txt unknown /proc/78/exe (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:kcompactd 78 root NOFD /proc/78/fd (opendir: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:ksmd 79 root cwd unknown /proc/79/cwd (readlink: Permission denied) 2024-05-08T05:56:40.853 INFO:tasks.workunit.client.0.smithi003.stdout:ksmd 79 root rtd unknown /proc/79/root (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:ksmd 79 root txt unknown /proc/79/exe (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:ksmd 79 root NOFD /proc/79/fd (opendir: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:khugepage 80 root cwd unknown /proc/80/cwd (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:khugepage 80 root rtd unknown /proc/80/root (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:khugepage 80 root txt unknown /proc/80/exe (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:khugepage 80 root NOFD /proc/80/fd (opendir: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:cryptd 81 root cwd unknown /proc/81/cwd (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:cryptd 81 root rtd unknown /proc/81/root (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:cryptd 81 root txt unknown /proc/81/exe (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:cryptd 81 root NOFD /proc/81/fd (opendir: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:kintegrit 82 root cwd unknown /proc/82/cwd (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:kintegrit 82 root rtd unknown /proc/82/root (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:kintegrit 82 root txt unknown /proc/82/exe (readlink: Permission denied) 2024-05-08T05:56:40.854 INFO:tasks.workunit.client.0.smithi003.stdout:kintegrit 82 root NOFD /proc/82/fd (opendir: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:kblockd 83 root cwd unknown /proc/83/cwd (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:kblockd 83 root rtd unknown /proc/83/root (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:kblockd 83 root txt unknown /proc/83/exe (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:kblockd 83 root NOFD /proc/83/fd (opendir: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:blkcg_pun 84 root cwd unknown /proc/84/cwd (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:blkcg_pun 84 root rtd unknown /proc/84/root (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:blkcg_pun 84 root txt unknown /proc/84/exe (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:blkcg_pun 84 root NOFD /proc/84/fd (opendir: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:tpm_dev_w 85 root cwd unknown /proc/85/cwd (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:tpm_dev_w 85 root rtd unknown /proc/85/root (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:tpm_dev_w 85 root txt unknown /proc/85/exe (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:tpm_dev_w 85 root NOFD /proc/85/fd (opendir: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:md 86 root cwd unknown /proc/86/cwd (readlink: Permission denied) 2024-05-08T05:56:40.855 INFO:tasks.workunit.client.0.smithi003.stdout:md 86 root rtd unknown /proc/86/root (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:md 86 root txt unknown /proc/86/exe (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:md 86 root NOFD /proc/86/fd (opendir: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:md_bitmap 87 root cwd unknown /proc/87/cwd (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:md_bitmap 87 root rtd unknown /proc/87/root (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:md_bitmap 87 root txt unknown /proc/87/exe (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:md_bitmap 87 root NOFD /proc/87/fd (opendir: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:edac-poll 88 root cwd unknown /proc/88/cwd (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:edac-poll 88 root rtd unknown /proc/88/root (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:edac-poll 88 root txt unknown /proc/88/exe (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:edac-poll 88 root NOFD /proc/88/fd (opendir: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:watchdogd 89 root cwd unknown /proc/89/cwd (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:watchdogd 89 root rtd unknown /proc/89/root (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:watchdogd 89 root txt unknown /proc/89/exe (readlink: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:watchdogd 89 root NOFD /proc/89/fd (opendir: Permission denied) 2024-05-08T05:56:40.856 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 91 root cwd unknown /proc/91/cwd (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 91 root rtd unknown /proc/91/root (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 91 root txt unknown /proc/91/exe (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 91 root NOFD /proc/91/fd (opendir: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kswapd0 93 root cwd unknown /proc/93/cwd (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kswapd0 93 root rtd unknown /proc/93/root (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kswapd0 93 root txt unknown /proc/93/exe (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kswapd0 93 root NOFD /proc/93/fd (opendir: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kthrotld 99 root cwd unknown /proc/99/cwd (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kthrotld 99 root rtd unknown /proc/99/root (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kthrotld 99 root txt unknown /proc/99/exe (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:kthrotld 99 root NOFD /proc/99/fd (opendir: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:acpi_ther 108 root cwd unknown /proc/108/cwd (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:acpi_ther 108 root rtd unknown /proc/108/root (readlink: Permission denied) 2024-05-08T05:56:40.857 INFO:tasks.workunit.client.0.smithi003.stdout:acpi_ther 108 root txt unknown /proc/108/exe (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:acpi_ther 108 root NOFD /proc/108/fd (opendir: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kmpath_rd 109 root cwd unknown /proc/109/cwd (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kmpath_rd 109 root rtd unknown /proc/109/root (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kmpath_rd 109 root txt unknown /proc/109/exe (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kmpath_rd 109 root NOFD /proc/109/fd (opendir: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kaluad 110 root cwd unknown /proc/110/cwd (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kaluad 110 root rtd unknown /proc/110/root (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kaluad 110 root txt unknown /proc/110/exe (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kaluad 110 root NOFD /proc/110/fd (opendir: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:mld 115 root cwd unknown /proc/115/cwd (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:mld 115 root rtd unknown /proc/115/root (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:mld 115 root txt unknown /proc/115/exe (readlink: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:mld 115 root NOFD /proc/115/fd (opendir: Permission denied) 2024-05-08T05:56:40.858 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 116 root cwd unknown /proc/116/cwd (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 116 root rtd unknown /proc/116/root (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 116 root txt unknown /proc/116/exe (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 116 root NOFD /proc/116/fd (opendir: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:ipv6_addr 117 root cwd unknown /proc/117/cwd (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:ipv6_addr 117 root rtd unknown /proc/117/root (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:ipv6_addr 117 root txt unknown /proc/117/exe (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:ipv6_addr 117 root NOFD /proc/117/fd (opendir: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kstrp 126 root cwd unknown /proc/126/cwd (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kstrp 126 root rtd unknown /proc/126/root (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kstrp 126 root txt unknown /proc/126/exe (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:kstrp 126 root NOFD /proc/126/fd (opendir: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:zswap-shr 132 root cwd unknown /proc/132/cwd (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:zswap-shr 132 root rtd unknown /proc/132/root (readlink: Permission denied) 2024-05-08T05:56:40.859 INFO:tasks.workunit.client.0.smithi003.stdout:zswap-shr 132 root txt unknown /proc/132/exe (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:zswap-shr 132 root NOFD /proc/132/fd (opendir: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 133 root cwd unknown /proc/133/cwd (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 133 root rtd unknown /proc/133/root (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 133 root txt unknown /proc/133/exe (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 133 root NOFD /proc/133/fd (opendir: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 279 root cwd unknown /proc/279/cwd (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 279 root rtd unknown /proc/279/root (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 279 root txt unknown /proc/279/exe (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 279 root NOFD /proc/279/fd (opendir: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 283 root cwd unknown /proc/283/cwd (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 283 root rtd unknown /proc/283/root (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 283 root txt unknown /proc/283/exe (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 283 root NOFD /proc/283/fd (opendir: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 286 root cwd unknown /proc/286/cwd (readlink: Permission denied) 2024-05-08T05:56:40.860 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 286 root rtd unknown /proc/286/root (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 286 root txt unknown /proc/286/exe (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 286 root NOFD /proc/286/fd (opendir: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 299 root cwd unknown /proc/299/cwd (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 299 root rtd unknown /proc/299/root (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 299 root txt unknown /proc/299/exe (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 299 root NOFD /proc/299/fd (opendir: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 358 root cwd unknown /proc/358/cwd (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 358 root rtd unknown /proc/358/root (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 358 root txt unknown /proc/358/exe (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 358 root NOFD /proc/358/fd (opendir: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 364 root cwd unknown /proc/364/cwd (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 364 root rtd unknown /proc/364/root (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 364 root txt unknown /proc/364/exe (readlink: Permission denied) 2024-05-08T05:56:40.861 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 364 root NOFD /proc/364/fd (opendir: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-wq 430 root cwd unknown /proc/430/cwd (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-wq 430 root rtd unknown /proc/430/root (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-wq 430 root txt unknown /proc/430/exe (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-wq 430 root NOFD /proc/430/fd (opendir: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-rese 436 root cwd unknown /proc/436/cwd (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-rese 436 root rtd unknown /proc/436/root (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-rese 436 root txt unknown /proc/436/exe (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-rese 436 root NOFD /proc/436/fd (opendir: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:ata_sff 437 root cwd unknown /proc/437/cwd (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:ata_sff 437 root rtd unknown /proc/437/root (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:ata_sff 437 root txt unknown /proc/437/exe (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:ata_sff 437 root NOFD /proc/437/fd (opendir: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-dele 440 root cwd unknown /proc/440/cwd (readlink: Permission denied) 2024-05-08T05:56:40.862 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-dele 440 root rtd unknown /proc/440/root (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-dele 440 root txt unknown /proc/440/exe (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-dele 440 root NOFD /proc/440/fd (opendir: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-auth 441 root cwd unknown /proc/441/cwd (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-auth 441 root rtd unknown /proc/441/root (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-auth 441 root txt unknown /proc/441/exe (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:nvme-auth 441 root NOFD /proc/441/fd (opendir: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:ixgbe 450 root cwd unknown /proc/450/cwd (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:ixgbe 450 root rtd unknown /proc/450/root (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:ixgbe 450 root txt unknown /proc/450/exe (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:ixgbe 450 root NOFD /proc/450/fd (opendir: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_0 452 root cwd unknown /proc/452/cwd (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_0 452 root rtd unknown /proc/452/root (readlink: Permission denied) 2024-05-08T05:56:40.863 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_0 452 root txt unknown /proc/452/exe (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_0 452 root NOFD /proc/452/fd (opendir: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 453 root cwd unknown /proc/453/cwd (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 453 root rtd unknown /proc/453/root (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 453 root txt unknown /proc/453/exe (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 453 root NOFD /proc/453/fd (opendir: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_1 454 root cwd unknown /proc/454/cwd (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_1 454 root rtd unknown /proc/454/root (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_1 454 root txt unknown /proc/454/exe (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_1 454 root NOFD /proc/454/fd (opendir: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 455 root cwd unknown /proc/455/cwd (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 455 root rtd unknown /proc/455/root (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 455 root txt unknown /proc/455/exe (readlink: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 455 root NOFD /proc/455/fd (opendir: Permission denied) 2024-05-08T05:56:40.864 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_2 456 root cwd unknown /proc/456/cwd (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_2 456 root rtd unknown /proc/456/root (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_2 456 root txt unknown /proc/456/exe (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_2 456 root NOFD /proc/456/fd (opendir: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 457 root cwd unknown /proc/457/cwd (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 457 root rtd unknown /proc/457/root (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 457 root txt unknown /proc/457/exe (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 457 root NOFD /proc/457/fd (opendir: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_3 458 root cwd unknown /proc/458/cwd (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_3 458 root rtd unknown /proc/458/root (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_3 458 root txt unknown /proc/458/exe (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_3 458 root NOFD /proc/458/fd (opendir: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 459 root cwd unknown /proc/459/cwd (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 459 root rtd unknown /proc/459/root (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 459 root txt unknown /proc/459/exe (readlink: Permission denied) 2024-05-08T05:56:40.865 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 459 root NOFD /proc/459/fd (opendir: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_4 460 root cwd unknown /proc/460/cwd (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_4 460 root rtd unknown /proc/460/root (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_4 460 root txt unknown /proc/460/exe (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_4 460 root NOFD /proc/460/fd (opendir: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 461 root cwd unknown /proc/461/cwd (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 461 root rtd unknown /proc/461/root (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 461 root txt unknown /proc/461/exe (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 461 root NOFD /proc/461/fd (opendir: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_5 462 root cwd unknown /proc/462/cwd (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_5 462 root rtd unknown /proc/462/root (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_5 462 root txt unknown /proc/462/exe (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_5 462 root NOFD /proc/462/fd (opendir: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 463 root cwd unknown /proc/463/cwd (readlink: Permission denied) 2024-05-08T05:56:40.866 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 463 root rtd unknown /proc/463/root (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 463 root txt unknown /proc/463/exe (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 463 root NOFD /proc/463/fd (opendir: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_6 464 root cwd unknown /proc/464/cwd (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_6 464 root rtd unknown /proc/464/root (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_6 464 root txt unknown /proc/464/exe (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_6 464 root NOFD /proc/464/fd (opendir: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 465 root cwd unknown /proc/465/cwd (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 465 root rtd unknown /proc/465/root (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 465 root txt unknown /proc/465/exe (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 465 root NOFD /proc/465/fd (opendir: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_7 466 root cwd unknown /proc/466/cwd (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_7 466 root rtd unknown /proc/466/root (readlink: Permission denied) 2024-05-08T05:56:40.867 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_7 466 root txt unknown /proc/466/exe (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_7 466 root NOFD /proc/466/fd (opendir: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 467 root cwd unknown /proc/467/cwd (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 467 root rtd unknown /proc/467/root (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 467 root txt unknown /proc/467/exe (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 467 root NOFD /proc/467/fd (opendir: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_8 468 root cwd unknown /proc/468/cwd (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_8 468 root rtd unknown /proc/468/root (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_8 468 root txt unknown /proc/468/exe (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_8 468 root NOFD /proc/468/fd (opendir: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 469 root cwd unknown /proc/469/cwd (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 469 root rtd unknown /proc/469/root (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 469 root txt unknown /proc/469/exe (readlink: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 469 root NOFD /proc/469/fd (opendir: Permission denied) 2024-05-08T05:56:40.868 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_9 470 root cwd unknown /proc/470/cwd (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_9 470 root rtd unknown /proc/470/root (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_9 470 root txt unknown /proc/470/exe (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_eh_9 470 root NOFD /proc/470/fd (opendir: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 471 root cwd unknown /proc/471/cwd (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 471 root rtd unknown /proc/471/root (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 471 root txt unknown /proc/471/exe (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:scsi_tmf_ 471 root NOFD /proc/471/fd (opendir: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:jbd2/sda1 500 root cwd unknown /proc/500/cwd (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:jbd2/sda1 500 root rtd unknown /proc/500/root (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:jbd2/sda1 500 root txt unknown /proc/500/exe (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:jbd2/sda1 500 root NOFD /proc/500/fd (opendir: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:ext4-rsv- 501 root cwd unknown /proc/501/cwd (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:ext4-rsv- 501 root rtd unknown /proc/501/root (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:ext4-rsv- 501 root txt unknown /proc/501/exe (readlink: Permission denied) 2024-05-08T05:56:40.869 INFO:tasks.workunit.client.0.smithi003.stdout:ext4-rsv- 501 root NOFD /proc/501/fd (opendir: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-j 576 root cwd unknown /proc/576/cwd (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-j 576 root rtd unknown /proc/576/root (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-j 576 root txt unknown /proc/576/exe (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-j 576 root NOFD /proc/576/fd (opendir: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:ext4lazyi 582 root cwd unknown /proc/582/cwd (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:ext4lazyi 582 root rtd unknown /proc/582/root (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:ext4lazyi 582 root txt unknown /proc/582/exe (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:ext4lazyi 582 root NOFD /proc/582/fd (opendir: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 584 root cwd unknown /proc/584/cwd (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 584 root rtd unknown /proc/584/root (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 584 root txt unknown /proc/584/exe (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 584 root NOFD /proc/584/fd (opendir: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-u 592 root cwd unknown /proc/592/cwd (readlink: Permission denied) 2024-05-08T05:56:40.870 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-u 592 root rtd unknown /proc/592/root (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-u 592 root txt unknown /proc/592/exe (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-u 592 root NOFD /proc/592/fd (opendir: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:ipmi-msgh 642 root cwd unknown /proc/642/cwd (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:ipmi-msgh 642 root rtd unknown /proc/642/root (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:ipmi-msgh 642 root txt unknown /proc/642/exe (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:ipmi-msgh 642 root NOFD /proc/642/fd (opendir: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:rpcbind 665 rpc cwd unknown /proc/665/cwd (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:rpcbind 665 rpc rtd unknown /proc/665/root (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:rpcbind 665 rpc txt unknown /proc/665/exe (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:rpcbind 665 rpc NOFD /proc/665/fd (opendir: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:kipmi0 666 root cwd unknown /proc/666/cwd (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:kipmi0 666 root rtd unknown /proc/666/root (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:kipmi0 666 root txt unknown /proc/666/exe (readlink: Permission denied) 2024-05-08T05:56:40.871 INFO:tasks.workunit.client.0.smithi003.stdout:kipmi0 666 root NOFD /proc/666/fd (opendir: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 root cwd unknown /proc/670/cwd (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 root rtd unknown /proc/670/root (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 root txt unknown /proc/670/exe (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 root NOFD /proc/670/fd (opendir: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 671 auditd root cwd unknown /proc/670/task/671/cwd (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 671 auditd root rtd unknown /proc/670/task/671/root (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 671 auditd root txt unknown /proc/670/task/671/exe (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 671 auditd root NOFD /proc/670/task/671/fd (opendir: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 673 auditd root cwd unknown /proc/670/task/673/cwd (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 673 auditd root rtd unknown /proc/670/task/673/root (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 673 auditd root txt unknown /proc/670/task/673/exe (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:auditd 670 673 auditd root NOFD /proc/670/task/673/fd (opendir: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:sedispatc 672 root cwd unknown /proc/672/cwd (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:sedispatc 672 root rtd unknown /proc/672/root (readlink: Permission denied) 2024-05-08T05:56:40.872 INFO:tasks.workunit.client.0.smithi003.stdout:sedispatc 672 root txt unknown /proc/672/exe (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:sedispatc 672 root NOFD /proc/672/fd (opendir: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:rpciod 674 root cwd unknown /proc/674/cwd (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:rpciod 674 root rtd unknown /proc/674/root (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:rpciod 674 root txt unknown /proc/674/exe (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:rpciod 674 root NOFD /proc/674/fd (opendir: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:xprtiod 675 root cwd unknown /proc/675/cwd (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:xprtiod 675 root rtd unknown /proc/675/root (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:xprtiod 675 root txt unknown /proc/675/exe (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:xprtiod 675 root NOFD /proc/675/fd (opendir: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 713 dbus cwd unknown /proc/713/cwd (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 713 dbus rtd unknown /proc/713/root (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 713 dbus txt unknown /proc/713/exe (readlink: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 713 dbus NOFD /proc/713/fd (opendir: Permission denied) 2024-05-08T05:56:40.873 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 714 dbus cwd unknown /proc/714/cwd (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 714 dbus rtd unknown /proc/714/root (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 714 dbus txt unknown /proc/714/exe (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:dbus-brok 714 dbus NOFD /proc/714/fd (opendir: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 root cwd unknown /proc/715/cwd (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 root rtd unknown /proc/715/root (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 root txt unknown /proc/715/exe (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 root NOFD /proc/715/fd (opendir: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 731 gmain root cwd unknown /proc/715/task/731/cwd (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 731 gmain root rtd unknown /proc/715/task/731/root (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 731 gmain root txt unknown /proc/715/task/731/exe (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 731 gmain root NOFD /proc/715/task/731/fd (opendir: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 732 gdbus root cwd unknown /proc/715/task/732/cwd (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 732 gdbus root rtd unknown /proc/715/task/732/root (readlink: Permission denied) 2024-05-08T05:56:40.874 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 732 gdbus root txt unknown /proc/715/task/732/exe (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:NetworkMa 715 732 gdbus root NOFD /proc/715/task/732/fd (opendir: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 root cwd unknown /proc/718/cwd (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 root rtd unknown /proc/718/root (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 root txt unknown /proc/718/exe (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 root NOFD /proc/718/fd (opendir: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 724 gmain root cwd unknown /proc/718/task/724/cwd (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 724 gmain root rtd unknown /proc/718/task/724/root (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 724 gmain root txt unknown /proc/718/task/724/exe (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:irqbalanc 718 724 gmain root NOFD /proc/718/task/724/fd (opendir: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:lsmd 719 libstoragemgmt cwd unknown /proc/719/cwd (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:lsmd 719 libstoragemgmt rtd unknown /proc/719/root (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:lsmd 719 libstoragemgmt txt unknown /proc/719/exe (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:lsmd 719 libstoragemgmt NOFD /proc/719/fd (opendir: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:mcelog 720 root cwd unknown /proc/720/cwd (readlink: Permission denied) 2024-05-08T05:56:40.875 INFO:tasks.workunit.client.0.smithi003.stdout:mcelog 720 root rtd unknown /proc/720/root (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:mcelog 720 root txt unknown /proc/720/exe (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:mcelog 720 root NOFD /proc/720/fd (opendir: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:smartd 721 root cwd unknown /proc/721/cwd (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:smartd 721 root rtd unknown /proc/721/root (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:smartd 721 root txt unknown /proc/721/exe (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:smartd 721 root NOFD /proc/721/fd (opendir: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-l 723 root cwd unknown /proc/723/cwd (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-l 723 root rtd unknown /proc/723/root (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-l 723 root txt unknown /proc/723/exe (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:systemd-l 723 root NOFD /proc/723/fd (opendir: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 763 root cwd unknown /proc/763/cwd (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 763 root rtd unknown /proc/763/root (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 763 root txt unknown /proc/763/exe (readlink: Permission denied) 2024-05-08T05:56:40.876 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 763 root NOFD /proc/763/fd (opendir: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 root cwd unknown /proc/767/cwd (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 root rtd unknown /proc/767/root (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 root txt unknown /proc/767/exe (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 root NOFD /proc/767/fd (opendir: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 768 gssproxy root cwd unknown /proc/767/task/768/cwd (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 768 gssproxy root rtd unknown /proc/767/task/768/root (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 768 gssproxy root txt unknown /proc/767/task/768/exe (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 768 gssproxy root NOFD /proc/767/task/768/fd (opendir: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 769 gssproxy root cwd unknown /proc/767/task/769/cwd (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 769 gssproxy root rtd unknown /proc/767/task/769/root (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 769 gssproxy root txt unknown /proc/767/task/769/exe (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 769 gssproxy root NOFD /proc/767/task/769/fd (opendir: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 770 gssproxy root cwd unknown /proc/767/task/770/cwd (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 770 gssproxy root rtd unknown /proc/767/task/770/root (readlink: Permission denied) 2024-05-08T05:56:40.877 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 770 gssproxy root txt unknown /proc/767/task/770/exe (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 770 gssproxy root NOFD /proc/767/task/770/fd (opendir: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 771 gssproxy root cwd unknown /proc/767/task/771/cwd (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 771 gssproxy root rtd unknown /proc/767/task/771/root (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 771 gssproxy root txt unknown /proc/767/task/771/exe (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 771 gssproxy root NOFD /proc/767/task/771/fd (opendir: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 772 gssproxy root cwd unknown /proc/767/task/772/cwd (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 772 gssproxy root rtd unknown /proc/767/task/772/root (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 772 gssproxy root txt unknown /proc/767/task/772/exe (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:gssproxy 767 772 gssproxy root NOFD /proc/767/task/772/fd (opendir: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:atd 774 root cwd unknown /proc/774/cwd (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:atd 774 root rtd unknown /proc/774/root (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:atd 774 root txt unknown /proc/774/exe (readlink: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:atd 774 root NOFD /proc/774/fd (opendir: Permission denied) 2024-05-08T05:56:40.878 INFO:tasks.workunit.client.0.smithi003.stdout:crond 775 root cwd unknown /proc/775/cwd (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:crond 775 root rtd unknown /proc/775/root (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:crond 775 root txt unknown /proc/775/exe (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:crond 775 root NOFD /proc/775/fd (opendir: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:nrpe 797 nrpe cwd unknown /proc/797/cwd (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:nrpe 797 nrpe rtd unknown /proc/797/root (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:nrpe 797 nrpe txt unknown /proc/797/exe (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:nrpe 797 nrpe NOFD /proc/797/fd (opendir: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1186 root cwd unknown /proc/1186/cwd (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1186 root rtd unknown /proc/1186/root (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1186 root txt unknown /proc/1186/exe (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1186 root NOFD /proc/1186/fd (opendir: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1190 root cwd unknown /proc/1190/cwd (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1190 root rtd unknown /proc/1190/root (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1190 root txt unknown /proc/1190/exe (readlink: Permission denied) 2024-05-08T05:56:40.879 INFO:tasks.workunit.client.0.smithi003.stdout:agetty 1190 root NOFD /proc/1190/fd (opendir: Permission denied) 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu cwd DIR 8,1 4096 2 / 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu rtd DIR 8,1 4096 2 / 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu txt REG 8,1 98216 18583 /usr/lib/systemd/systemd 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu DEL REG 8,1 596 /etc/selinux/targeted/contexts/files/file_contexts.bin 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 914368 7110 /usr/lib64/libm.so.6 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 1714208 7271 /usr/lib64/libp11-kit.so.0.3.1 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu DEL REG 8,1 7490 /usr/lib64/libcrypto.so.3.0.7 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu DEL REG 8,1 1130 /etc/selinux/targeted/contexts/files/file_contexts.homedirs.bin 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 153600 7292 /usr/lib64/libgpg-error.so.0.32.0 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 178728 7177 /usr/lib64/liblzma.so.5.2.5 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 1288504 7366 /usr/lib64/libgcrypt.so.20.4.0 2024-05-08T05:56:40.880 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 44784 7268 /usr/lib64/libffi.so.8.1.0 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 28552 7287 /usr/lib64/libattr.so.1.1.2501 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 882376 7182 /usr/lib64/libzstd.so.1.5.1 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 144120 7350 /usr/lib64/liblz4.so.1.9.3 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 201808 7179 /usr/lib64/libcrypt.so.2.0.0 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 229432 7609 /usr/lib64/libblkid.so.1.1.0 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 291512 7639 /usr/lib64/libmount.so.1.1.0 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 3785856 46432 /usr/lib64/systemd/libsystemd-shared-252.so 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 32528 7206 /usr/lib64/libcap-ng.so.0.0.0 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 40424 8439 /usr/lib64/libeconf.so.0.4.1 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 36296 7185 /usr/lib64/libcap.so.2.48 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 107424 7663 /usr/lib64/libkmod.so.2.3.6 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 138848 7210 /usr/lib64/libaudit.so.1.0.0 2024-05-08T05:56:40.881 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 2157904 46429 /usr/lib64/systemd/libsystemd-core-252.so 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 40496 7288 /usr/lib64/libacl.so.1.1.2301 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 69592 8448 /usr/lib64/libpam.so.0.85.1 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 127680 7455 /usr/lib64/libseccomp.so.2.5.2 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 108136 10561 /usr/lib64/libgcc_s-11-20231218.so.1 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu mem REG 0,19 0 19 /sys/fs/selinux/status 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 0r CHR 1,3 0t0 4 /dev/null 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 1u unix 0x0000000000000000 0t0 25682 type=STREAM (CONNECTED) 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 2u unix 0x0000000000000000 0t0 25682 type=STREAM (CONNECTED) 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 3u unix 0x0000000000000000 0t0 25710 type=DGRAM (CONNECTED) 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 4u a_inode 0,14 0 2069 [eventpoll:5,6,8,9,11,12,14,15,16,17,19,20,21] 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 5u a_inode 0,14 0 2069 [signalfd] 2024-05-08T05:56:40.882 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 6r a_inode 0,14 0 2069 inotify 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 7r DIR 0,25 0 3321 /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 8u a_inode 0,14 0 2069 [timerfd] 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 9u a_inode 0,14 0 2069 [eventpoll:10,13] 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 10r REG 0,20 0 25699 /proc/4813/mountinfo 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 11r a_inode 0,14 0 2069 inotify 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 12u unix 0x0000000000000000 0t0 25732 /run/user/1000/bus type=STREAM (LISTEN) 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 13r a_inode 0,14 0 2069 inotify 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 14r REG 0,20 0 4026532076 /proc/swaps 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 15u netlink 0t0 25714 KOBJECT_UEVENT 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 16u unix 0x0000000000000000 0t0 25719 /run/user/1000/systemd/notify type=DGRAM (UNCONNECTED) 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 17u unix 0x0000000000000000 0t0 25720 type=DGRAM (CONNECTED) 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 18u unix 0x0000000000000000 0t0 25721 type=DGRAM (CONNECTED) 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 19u unix 0x0000000000000000 0t0 25722 /run/user/1000/systemd/private type=STREAM (LISTEN) 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 20u unix 0x0000000000000000 0t0 25724 type=STREAM (CONNECTED) 2024-05-08T05:56:40.883 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 4813 ubuntu 21u a_inode 0,14 0 2069 [timerfd] 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 4815 ubuntu cwd unknown /proc/4815/cwd (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 4815 ubuntu rtd unknown /proc/4815/root (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 4815 ubuntu txt unknown /proc/4815/exe (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 4815 ubuntu NOFD /proc/4815/fd (opendir: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 root cwd unknown /proc/5058/cwd (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 root rtd unknown /proc/5058/root (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 root txt unknown /proc/5058/exe (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 root NOFD /proc/5058/fd (opendir: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5060 in:imjour root cwd unknown /proc/5058/task/5060/cwd (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5060 in:imjour root rtd unknown /proc/5058/task/5060/root (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5060 in:imjour root txt unknown /proc/5058/task/5060/exe (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5060 in:imjour root NOFD /proc/5058/task/5060/fd (opendir: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5061 rs:main root cwd unknown /proc/5058/task/5061/cwd (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5061 rs:main root rtd unknown /proc/5058/task/5061/root (readlink: Permission denied) 2024-05-08T05:56:40.884 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5061 rs:main root txt unknown /proc/5058/task/5061/exe (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:rsyslogd 5058 5061 rs:main root NOFD /proc/5058/task/5061/fd (opendir: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:tls-strp 16513 root cwd unknown /proc/16513/cwd (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:tls-strp 16513 root rtd unknown /proc/16513/root (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:tls-strp 16513 root txt unknown /proc/16513/exe (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:tls-strp 16513 root NOFD /proc/16513/fd (opendir: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 polkitd cwd unknown /proc/16525/cwd (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 polkitd rtd unknown /proc/16525/root (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 polkitd txt unknown /proc/16525/exe (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 polkitd NOFD /proc/16525/fd (opendir: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16527 gmain polkitd cwd unknown /proc/16525/task/16527/cwd (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16527 gmain polkitd rtd unknown /proc/16525/task/16527/root (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16527 gmain polkitd txt unknown /proc/16525/task/16527/exe (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16527 gmain polkitd NOFD /proc/16525/task/16527/fd (opendir: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16528 gdbus polkitd cwd unknown /proc/16525/task/16528/cwd (readlink: Permission denied) 2024-05-08T05:56:40.885 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16528 gdbus polkitd rtd unknown /proc/16525/task/16528/root (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16528 gdbus polkitd txt unknown /proc/16525/task/16528/exe (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16528 gdbus polkitd NOFD /proc/16525/task/16528/fd (opendir: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16529 JS\x20Hel polkitd cwd unknown /proc/16525/task/16529/cwd (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16529 JS\x20Hel polkitd rtd unknown /proc/16525/task/16529/root (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16529 JS\x20Hel polkitd txt unknown /proc/16525/task/16529/exe (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16529 JS\x20Hel polkitd NOFD /proc/16525/task/16529/fd (opendir: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16530 JS\x20Hel polkitd cwd unknown /proc/16525/task/16530/cwd (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16530 JS\x20Hel polkitd rtd unknown /proc/16525/task/16530/root (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16530 JS\x20Hel polkitd txt unknown /proc/16525/task/16530/exe (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16530 JS\x20Hel polkitd NOFD /proc/16525/task/16530/fd (opendir: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16531 JS\x20Hel polkitd cwd unknown /proc/16525/task/16531/cwd (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16531 JS\x20Hel polkitd rtd unknown /proc/16525/task/16531/root (readlink: Permission denied) 2024-05-08T05:56:40.886 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16531 JS\x20Hel polkitd txt unknown /proc/16525/task/16531/exe (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16531 JS\x20Hel polkitd NOFD /proc/16525/task/16531/fd (opendir: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16532 JS\x20Hel polkitd cwd unknown /proc/16525/task/16532/cwd (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16532 JS\x20Hel polkitd rtd unknown /proc/16525/task/16532/root (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16532 JS\x20Hel polkitd txt unknown /proc/16525/task/16532/exe (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16532 JS\x20Hel polkitd NOFD /proc/16525/task/16532/fd (opendir: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16533 JS\x20Hel polkitd cwd unknown /proc/16525/task/16533/cwd (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16533 JS\x20Hel polkitd rtd unknown /proc/16525/task/16533/root (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16533 JS\x20Hel polkitd txt unknown /proc/16525/task/16533/exe (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16533 JS\x20Hel polkitd NOFD /proc/16525/task/16533/fd (opendir: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16534 JS\x20Hel polkitd cwd unknown /proc/16525/task/16534/cwd (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16534 JS\x20Hel polkitd rtd unknown /proc/16525/task/16534/root (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16534 JS\x20Hel polkitd txt unknown /proc/16525/task/16534/exe (readlink: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16534 JS\x20Hel polkitd NOFD /proc/16525/task/16534/fd (opendir: Permission denied) 2024-05-08T05:56:40.887 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16535 JS\x20Hel polkitd cwd unknown /proc/16525/task/16535/cwd (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16535 JS\x20Hel polkitd rtd unknown /proc/16525/task/16535/root (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16535 JS\x20Hel polkitd txt unknown /proc/16525/task/16535/exe (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16535 JS\x20Hel polkitd NOFD /proc/16525/task/16535/fd (opendir: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16536 JS\x20Hel polkitd cwd unknown /proc/16525/task/16536/cwd (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16536 JS\x20Hel polkitd rtd unknown /proc/16525/task/16536/root (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16536 JS\x20Hel polkitd txt unknown /proc/16525/task/16536/exe (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16536 JS\x20Hel polkitd NOFD /proc/16525/task/16536/fd (opendir: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16537 polkitd polkitd cwd unknown /proc/16525/task/16537/cwd (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16537 polkitd polkitd rtd unknown /proc/16525/task/16537/root (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16537 polkitd polkitd txt unknown /proc/16525/task/16537/exe (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:polkitd 16525 16537 polkitd polkitd NOFD /proc/16525/task/16537/fd (opendir: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 16966 root cwd unknown /proc/16966/cwd (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 16966 root rtd unknown /proc/16966/root (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 16966 root txt unknown /proc/16966/exe (readlink: Permission denied) 2024-05-08T05:56:40.888 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 16966 root NOFD /proc/16966/fd (opendir: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20383 root cwd unknown /proc/20383/cwd (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20383 root rtd unknown /proc/20383/root (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20383 root txt unknown /proc/20383/exe (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20383 root NOFD /proc/20383/fd (opendir: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20412 root cwd unknown /proc/20412/cwd (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20412 root rtd unknown /proc/20412/root (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20412 root txt unknown /proc/20412/exe (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20412 root NOFD /proc/20412/fd (opendir: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20441 root cwd unknown /proc/20441/cwd (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20441 root rtd unknown /proc/20441/root (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20441 root txt unknown /proc/20441/exe (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20441 root NOFD /proc/20441/fd (opendir: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20470 root cwd unknown /proc/20470/cwd (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20470 root rtd unknown /proc/20470/root (readlink: Permission denied) 2024-05-08T05:56:40.889 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20470 root txt unknown /proc/20470/exe (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20470 root NOFD /proc/20470/fd (opendir: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20499 root cwd unknown /proc/20499/cwd (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20499 root rtd unknown /proc/20499/root (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20499 root txt unknown /proc/20499/exe (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:kdmflush/ 20499 root NOFD /proc/20499/fd (opendir: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfsalloc 20663 root cwd unknown /proc/20663/cwd (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfsalloc 20663 root rtd unknown /proc/20663/root (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfsalloc 20663 root txt unknown /proc/20663/exe (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfsalloc 20663 root NOFD /proc/20663/fd (opendir: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfs_mru_c 20664 root cwd unknown /proc/20664/cwd (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfs_mru_c 20664 root rtd unknown /proc/20664/root (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfs_mru_c 20664 root txt unknown /proc/20664/exe (readlink: Permission denied) 2024-05-08T05:56:40.890 INFO:tasks.workunit.client.0.smithi003.stdout:xfs_mru_c 20664 root NOFD /proc/20664/fd (opendir: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-buf/d 20665 root cwd unknown /proc/20665/cwd (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-buf/d 20665 root rtd unknown /proc/20665/root (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-buf/d 20665 root txt unknown /proc/20665/exe (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-buf/d 20665 root NOFD /proc/20665/fd (opendir: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-conv/ 20666 root cwd unknown /proc/20666/cwd (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-conv/ 20666 root rtd unknown /proc/20666/root (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-conv/ 20666 root txt unknown /proc/20666/exe (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-conv/ 20666 root NOFD /proc/20666/fd (opendir: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-recla 20667 root cwd unknown /proc/20667/cwd (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-recla 20667 root rtd unknown /proc/20667/root (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-recla 20667 root txt unknown /proc/20667/exe (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-recla 20667 root NOFD /proc/20667/fd (opendir: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-block 20668 root cwd unknown /proc/20668/cwd (readlink: Permission denied) 2024-05-08T05:56:40.891 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-block 20668 root rtd unknown /proc/20668/root (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-block 20668 root txt unknown /proc/20668/exe (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-block 20668 root NOFD /proc/20668/fd (opendir: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-inode 20669 root cwd unknown /proc/20669/cwd (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-inode 20669 root rtd unknown /proc/20669/root (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-inode 20669 root txt unknown /proc/20669/exe (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-inode 20669 root NOFD /proc/20669/fd (opendir: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-log/d 20670 root cwd unknown /proc/20670/cwd (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-log/d 20670 root rtd unknown /proc/20670/root (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-log/d 20670 root txt unknown /proc/20670/exe (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-log/d 20670 root NOFD /proc/20670/fd (opendir: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-cil/d 20671 root cwd unknown /proc/20671/cwd (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-cil/d 20671 root rtd unknown /proc/20671/root (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-cil/d 20671 root txt unknown /proc/20671/exe (readlink: Permission denied) 2024-05-08T05:56:40.892 INFO:tasks.workunit.client.0.smithi003.stdout:xfs-cil/d 20671 root NOFD /proc/20671/fd (opendir: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:xfsaild/d 20672 root cwd unknown /proc/20672/cwd (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:xfsaild/d 20672 root rtd unknown /proc/20672/root (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:xfsaild/d 20672 root txt unknown /proc/20672/exe (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:xfsaild/d 20672 root NOFD /proc/20672/fd (opendir: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21601 root cwd unknown /proc/21601/cwd (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21601 root rtd unknown /proc/21601/root (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21601 root txt unknown /proc/21601/exe (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21601 root NOFD /proc/21601/fd (opendir: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21604 ubuntu cwd unknown /proc/21604/cwd (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21604 ubuntu rtd unknown /proc/21604/root (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21604 ubuntu txt unknown /proc/21604/exe (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 21604 ubuntu NOFD /proc/21604/fd (opendir: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:chronyd 21664 chrony cwd unknown /proc/21664/cwd (readlink: Permission denied) 2024-05-08T05:56:40.893 INFO:tasks.workunit.client.0.smithi003.stdout:chronyd 21664 chrony rtd unknown /proc/21664/root (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:chronyd 21664 chrony txt unknown /proc/21664/exe (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:chronyd 21664 chrony NOFD /proc/21664/fd (opendir: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 21929 root cwd unknown /proc/21929/cwd (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 21929 root rtd unknown /proc/21929/root (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 21929 root txt unknown /proc/21929/exe (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 21929 root NOFD /proc/21929/fd (opendir: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-cras 21979 ceph cwd unknown /proc/21979/cwd (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-cras 21979 ceph rtd unknown /proc/21979/root (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-cras 21979 ceph txt unknown /proc/21979/exe (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-cras 21979 ceph NOFD /proc/21979/fd (opendir: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 22511 root cwd unknown /proc/22511/cwd (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 22511 root rtd unknown /proc/22511/root (readlink: Permission denied) 2024-05-08T05:56:40.894 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 22511 root txt unknown /proc/22511/exe (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 22511 root NOFD /proc/22511/fd (opendir: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 23559 root cwd unknown /proc/23559/cwd (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 23559 root rtd unknown /proc/23559/root (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 23559 root txt unknown /proc/23559/exe (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 23559 root NOFD /proc/23559/fd (opendir: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 28732 root cwd unknown /proc/28732/cwd (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 28732 root rtd unknown /proc/28732/root (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 28732 root txt unknown /proc/28732/exe (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 28732 root NOFD /proc/28732/fd (opendir: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 28991 root cwd unknown /proc/28991/cwd (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 28991 root rtd unknown /proc/28991/root (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 28991 root txt unknown /proc/28991/exe (readlink: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 28991 root NOFD /proc/28991/fd (opendir: Permission denied) 2024-05-08T05:56:40.895 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29118 root cwd unknown /proc/29118/cwd (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29118 root rtd unknown /proc/29118/root (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29118 root txt unknown /proc/29118/exe (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29118 root NOFD /proc/29118/fd (opendir: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29135 root cwd unknown /proc/29135/cwd (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29135 root rtd unknown /proc/29135/root (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29135 root txt unknown /proc/29135/exe (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29135 root NOFD /proc/29135/fd (opendir: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29136 root cwd unknown /proc/29136/cwd (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29136 root rtd unknown /proc/29136/root (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29136 root txt unknown /proc/29136/exe (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 29136 root NOFD /proc/29136/fd (opendir: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29153 root cwd unknown /proc/29153/cwd (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29153 root rtd unknown /proc/29153/root (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29153 root txt unknown /proc/29153/exe (readlink: Permission denied) 2024-05-08T05:56:40.896 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 29153 root NOFD /proc/29153/fd (opendir: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 29288 root cwd unknown /proc/29288/cwd (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 29288 root rtd unknown /proc/29288/root (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 29288 root txt unknown /proc/29288/exe (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 29288 root NOFD /proc/29288/fd (opendir: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 29400 root cwd unknown /proc/29400/cwd (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 29400 root rtd unknown /proc/29400/root (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 29400 root txt unknown /proc/29400/exe (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 29400 root NOFD /proc/29400/fd (opendir: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30133 root cwd unknown /proc/30133/cwd (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30133 root rtd unknown /proc/30133/root (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30133 root txt unknown /proc/30133/exe (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30133 root NOFD /proc/30133/fd (opendir: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30135 root cwd unknown /proc/30135/cwd (readlink: Permission denied) 2024-05-08T05:56:40.897 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30135 root rtd unknown /proc/30135/root (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30135 root txt unknown /proc/30135/exe (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30135 root NOFD /proc/30135/fd (opendir: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 ceph cwd unknown /proc/30137/cwd (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 ceph rtd unknown /proc/30137/root (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 ceph txt unknown /proc/30137/exe (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 ceph NOFD /proc/30137/fd (opendir: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30138 log ceph cwd unknown /proc/30137/task/30138/cwd (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30138 log ceph rtd unknown /proc/30137/task/30138/root (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30138 log ceph txt unknown /proc/30137/task/30138/exe (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30138 log ceph NOFD /proc/30137/task/30138/fd (opendir: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30139 service ceph cwd unknown /proc/30137/task/30139/cwd (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30139 service ceph rtd unknown /proc/30137/task/30139/root (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30139 service ceph txt unknown /proc/30137/task/30139/exe (readlink: Permission denied) 2024-05-08T05:56:40.898 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30139 service ceph NOFD /proc/30137/task/30139/fd (opendir: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30140 admin_soc ceph cwd unknown /proc/30137/task/30140/cwd (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30140 admin_soc ceph rtd unknown /proc/30137/task/30140/root (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30140 admin_soc ceph txt unknown /proc/30137/task/30140/exe (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30140 admin_soc ceph NOFD /proc/30137/task/30140/fd (opendir: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30141 signal_ha ceph cwd unknown /proc/30137/task/30141/cwd (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30141 signal_ha ceph rtd unknown /proc/30137/task/30141/root (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30141 signal_ha ceph txt unknown /proc/30137/task/30141/exe (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30141 signal_ha ceph NOFD /proc/30137/task/30141/fd (opendir: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30142 rocksdb:l ceph cwd unknown /proc/30137/task/30142/cwd (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30142 rocksdb:l ceph rtd unknown /proc/30137/task/30142/root (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30142 rocksdb:l ceph txt unknown /proc/30137/task/30142/exe (readlink: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30142 rocksdb:l ceph NOFD /proc/30137/task/30142/fd (opendir: Permission denied) 2024-05-08T05:56:40.899 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30143 rocksdb:h ceph cwd unknown /proc/30137/task/30143/cwd (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30143 rocksdb:h ceph rtd unknown /proc/30137/task/30143/root (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30143 rocksdb:h ceph txt unknown /proc/30137/task/30143/exe (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30143 rocksdb:h ceph NOFD /proc/30137/task/30143/fd (opendir: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30159 ceph-mon ceph cwd unknown /proc/30137/task/30159/cwd (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30159 ceph-mon ceph rtd unknown /proc/30137/task/30159/root (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30159 ceph-mon ceph txt unknown /proc/30137/task/30159/exe (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30159 ceph-mon ceph NOFD /proc/30137/task/30159/fd (opendir: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30160 fn_monsto ceph cwd unknown /proc/30137/task/30160/cwd (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30160 fn_monsto ceph rtd unknown /proc/30137/task/30160/root (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30160 fn_monsto ceph txt unknown /proc/30137/task/30160/exe (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30160 fn_monsto ceph NOFD /proc/30137/task/30160/fd (opendir: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30161 msgr-work ceph cwd unknown /proc/30137/task/30161/cwd (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30161 msgr-work ceph rtd unknown /proc/30137/task/30161/root (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30161 msgr-work ceph txt unknown /proc/30137/task/30161/exe (readlink: Permission denied) 2024-05-08T05:56:40.900 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30161 msgr-work ceph NOFD /proc/30137/task/30161/fd (opendir: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30162 msgr-work ceph cwd unknown /proc/30137/task/30162/cwd (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30162 msgr-work ceph rtd unknown /proc/30137/task/30162/root (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30162 msgr-work ceph txt unknown /proc/30137/task/30162/exe (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30162 msgr-work ceph NOFD /proc/30137/task/30162/fd (opendir: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30163 msgr-work ceph cwd unknown /proc/30137/task/30163/cwd (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30163 msgr-work ceph rtd unknown /proc/30137/task/30163/root (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30163 msgr-work ceph txt unknown /proc/30137/task/30163/exe (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30163 msgr-work ceph NOFD /proc/30137/task/30163/fd (opendir: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30164 OpHistory ceph cwd unknown /proc/30137/task/30164/cwd (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30164 OpHistory ceph rtd unknown /proc/30137/task/30164/root (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30164 OpHistory ceph txt unknown /proc/30137/task/30164/exe (readlink: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30164 OpHistory ceph NOFD /proc/30137/task/30164/fd (opendir: Permission denied) 2024-05-08T05:56:40.901 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30165 fin ceph cwd unknown /proc/30137/task/30165/cwd (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30165 fin ceph rtd unknown /proc/30137/task/30165/root (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30165 fin ceph txt unknown /proc/30137/task/30165/exe (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30165 fin ceph NOFD /proc/30137/task/30165/fd (opendir: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30166 safe_time ceph cwd unknown /proc/30137/task/30166/cwd (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30166 safe_time ceph rtd unknown /proc/30137/task/30166/root (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30166 safe_time ceph txt unknown /proc/30137/task/30166/exe (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30166 safe_time ceph NOFD /proc/30137/task/30166/fd (opendir: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30167 cpu_tp ceph cwd unknown /proc/30137/task/30167/cwd (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30167 cpu_tp ceph rtd unknown /proc/30137/task/30167/root (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30167 cpu_tp ceph txt unknown /proc/30137/task/30167/exe (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30167 cpu_tp ceph NOFD /proc/30137/task/30167/fd (opendir: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30168 cpu_tp ceph cwd unknown /proc/30137/task/30168/cwd (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30168 cpu_tp ceph rtd unknown /proc/30137/task/30168/root (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30168 cpu_tp ceph txt unknown /proc/30137/task/30168/exe (readlink: Permission denied) 2024-05-08T05:56:40.902 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30168 cpu_tp ceph NOFD /proc/30137/task/30168/fd (opendir: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30169 cpu_tp ceph cwd unknown /proc/30137/task/30169/cwd (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30169 cpu_tp ceph rtd unknown /proc/30137/task/30169/root (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30169 cpu_tp ceph txt unknown /proc/30137/task/30169/exe (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30169 cpu_tp ceph NOFD /proc/30137/task/30169/fd (opendir: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30170 cpu_tp ceph cwd unknown /proc/30137/task/30170/cwd (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30170 cpu_tp ceph rtd unknown /proc/30137/task/30170/root (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30170 cpu_tp ceph txt unknown /proc/30137/task/30170/exe (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30170 cpu_tp ceph NOFD /proc/30137/task/30170/fd (opendir: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30171 ms_dispat ceph cwd unknown /proc/30137/task/30171/cwd (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30171 ms_dispat ceph rtd unknown /proc/30137/task/30171/root (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30171 ms_dispat ceph txt unknown /proc/30137/task/30171/exe (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30171 ms_dispat ceph NOFD /proc/30137/task/30171/fd (opendir: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30172 ms_local ceph cwd unknown /proc/30137/task/30172/cwd (readlink: Permission denied) 2024-05-08T05:56:40.903 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30172 ms_local ceph rtd unknown /proc/30137/task/30172/root (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30172 ms_local ceph txt unknown /proc/30137/task/30172/exe (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30172 ms_local ceph NOFD /proc/30137/task/30172/fd (opendir: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30173 safe_time ceph cwd unknown /proc/30137/task/30173/cwd (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30173 safe_time ceph rtd unknown /proc/30137/task/30173/root (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30173 safe_time ceph txt unknown /proc/30137/task/30173/exe (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30173 safe_time ceph NOFD /proc/30137/task/30173/fd (opendir: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30174 ms_dispat ceph cwd unknown /proc/30137/task/30174/cwd (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30174 ms_dispat ceph rtd unknown /proc/30137/task/30174/root (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30174 ms_dispat ceph txt unknown /proc/30137/task/30174/exe (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30174 ms_dispat ceph NOFD /proc/30137/task/30174/fd (opendir: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30175 ms_local ceph cwd unknown /proc/30137/task/30175/cwd (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30175 ms_local ceph rtd unknown /proc/30137/task/30175/root (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30175 ms_local ceph txt unknown /proc/30137/task/30175/exe (readlink: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 30175 ms_local ceph NOFD /proc/30137/task/30175/fd (opendir: Permission denied) 2024-05-08T05:56:40.904 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 69602 rstore_co ceph cwd unknown /proc/30137/task/69602/cwd (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 69602 rstore_co ceph rtd unknown /proc/30137/task/69602/root (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 69602 rstore_co ceph txt unknown /proc/30137/task/69602/exe (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mon 30137 69602 rstore_co ceph NOFD /proc/30137/task/69602/fd (opendir: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30370 root cwd unknown /proc/30370/cwd (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30370 root rtd unknown /proc/30370/root (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30370 root txt unknown /proc/30370/exe (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 30370 root NOFD /proc/30370/fd (opendir: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30372 root cwd unknown /proc/30372/cwd (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30372 root rtd unknown /proc/30372/root (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30372 root txt unknown /proc/30372/exe (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 30372 root NOFD /proc/30372/fd (opendir: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 ceph cwd unknown /proc/30374/cwd (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 ceph rtd unknown /proc/30374/root (readlink: Permission denied) 2024-05-08T05:56:40.905 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 ceph txt unknown /proc/30374/exe (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 ceph NOFD /proc/30374/fd (opendir: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87876 log ceph cwd unknown /proc/30374/task/87876/cwd (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87876 log ceph rtd unknown /proc/30374/task/87876/root (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87876 log ceph txt unknown /proc/30374/task/87876/exe (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87876 log ceph NOFD /proc/30374/task/87876/fd (opendir: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87878 msgr-work ceph cwd unknown /proc/30374/task/87878/cwd (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87878 msgr-work ceph rtd unknown /proc/30374/task/87878/root (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87878 msgr-work ceph txt unknown /proc/30374/task/87878/exe (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87878 msgr-work ceph NOFD /proc/30374/task/87878/fd (opendir: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87879 msgr-work ceph cwd unknown /proc/30374/task/87879/cwd (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87879 msgr-work ceph rtd unknown /proc/30374/task/87879/root (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87879 msgr-work ceph txt unknown /proc/30374/task/87879/exe (readlink: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87879 msgr-work ceph NOFD /proc/30374/task/87879/fd (opendir: Permission denied) 2024-05-08T05:56:40.906 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87880 msgr-work ceph cwd unknown /proc/30374/task/87880/cwd (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87880 msgr-work ceph rtd unknown /proc/30374/task/87880/root (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87880 msgr-work ceph txt unknown /proc/30374/task/87880/exe (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87880 msgr-work ceph NOFD /proc/30374/task/87880/fd (opendir: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87884 service ceph cwd unknown /proc/30374/task/87884/cwd (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87884 service ceph rtd unknown /proc/30374/task/87884/root (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87884 service ceph txt unknown /proc/30374/task/87884/exe (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87884 service ceph NOFD /proc/30374/task/87884/fd (opendir: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87885 admin_soc ceph cwd unknown /proc/30374/task/87885/cwd (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87885 admin_soc ceph rtd unknown /proc/30374/task/87885/root (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87885 admin_soc ceph txt unknown /proc/30374/task/87885/exe (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87885 admin_soc ceph NOFD /proc/30374/task/87885/fd (opendir: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87886 ceph_time ceph cwd unknown /proc/30374/task/87886/cwd (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87886 ceph_time ceph rtd unknown /proc/30374/task/87886/root (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87886 ceph_time ceph txt unknown /proc/30374/task/87886/exe (readlink: Permission denied) 2024-05-08T05:56:40.907 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87886 ceph_time ceph NOFD /proc/30374/task/87886/fd (opendir: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87887 fn_anonym ceph cwd unknown /proc/30374/task/87887/cwd (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87887 fn_anonym ceph rtd unknown /proc/30374/task/87887/root (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87887 fn_anonym ceph txt unknown /proc/30374/task/87887/exe (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87887 fn_anonym ceph NOFD /proc/30374/task/87887/fd (opendir: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87888 signal_ha ceph cwd unknown /proc/30374/task/87888/cwd (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87888 signal_ha ceph rtd unknown /proc/30374/task/87888/root (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87888 signal_ha ceph txt unknown /proc/30374/task/87888/exe (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87888 signal_ha ceph NOFD /proc/30374/task/87888/fd (opendir: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87889 mgrsb-fin ceph cwd unknown /proc/30374/task/87889/cwd (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87889 mgrsb-fin ceph rtd unknown /proc/30374/task/87889/root (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87889 mgrsb-fin ceph txt unknown /proc/30374/task/87889/exe (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87889 mgrsb-fin ceph NOFD /proc/30374/task/87889/fd (opendir: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87890 ms_dispat ceph cwd unknown /proc/30374/task/87890/cwd (readlink: Permission denied) 2024-05-08T05:56:40.908 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87890 ms_dispat ceph rtd unknown /proc/30374/task/87890/root (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87890 ms_dispat ceph txt unknown /proc/30374/task/87890/exe (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87890 ms_dispat ceph NOFD /proc/30374/task/87890/fd (opendir: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87891 ms_local ceph cwd unknown /proc/30374/task/87891/cwd (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87891 ms_local ceph rtd unknown /proc/30374/task/87891/root (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87891 ms_local ceph txt unknown /proc/30374/task/87891/exe (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87891 ms_local ceph NOFD /proc/30374/task/87891/fd (opendir: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87892 io_contex ceph cwd unknown /proc/30374/task/87892/cwd (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87892 io_contex ceph rtd unknown /proc/30374/task/87892/root (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87892 io_contex ceph txt unknown /proc/30374/task/87892/exe (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87892 io_contex ceph NOFD /proc/30374/task/87892/fd (opendir: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87893 io_contex ceph cwd unknown /proc/30374/task/87893/cwd (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87893 io_contex ceph rtd unknown /proc/30374/task/87893/root (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87893 io_contex ceph txt unknown /proc/30374/task/87893/exe (readlink: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87893 io_contex ceph NOFD /proc/30374/task/87893/fd (opendir: Permission denied) 2024-05-08T05:56:40.909 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87894 safe_time ceph cwd unknown /proc/30374/task/87894/cwd (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87894 safe_time ceph rtd unknown /proc/30374/task/87894/root (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87894 safe_time ceph txt unknown /proc/30374/task/87894/exe (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87894 safe_time ceph NOFD /proc/30374/task/87894/fd (opendir: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87895 safe_time ceph cwd unknown /proc/30374/task/87895/cwd (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87895 safe_time ceph rtd unknown /proc/30374/task/87895/root (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87895 safe_time ceph txt unknown /proc/30374/task/87895/exe (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87895 safe_time ceph NOFD /proc/30374/task/87895/fd (opendir: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87896 safe_time ceph cwd unknown /proc/30374/task/87896/cwd (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87896 safe_time ceph rtd unknown /proc/30374/task/87896/root (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87896 safe_time ceph txt unknown /proc/30374/task/87896/exe (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87896 safe_time ceph NOFD /proc/30374/task/87896/fd (opendir: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87897 fn_anonym ceph cwd unknown /proc/30374/task/87897/cwd (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87897 fn_anonym ceph rtd unknown /proc/30374/task/87897/root (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87897 fn_anonym ceph txt unknown /proc/30374/task/87897/exe (readlink: Permission denied) 2024-05-08T05:56:40.910 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87897 fn_anonym ceph NOFD /proc/30374/task/87897/fd (opendir: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87898 flusher ceph cwd unknown /proc/30374/task/87898/cwd (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87898 flusher ceph rtd unknown /proc/30374/task/87898/root (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87898 flusher ceph txt unknown /proc/30374/task/87898/exe (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87898 flusher ceph NOFD /proc/30374/task/87898/fd (opendir: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87899 safe_time ceph cwd unknown /proc/30374/task/87899/cwd (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87899 safe_time ceph rtd unknown /proc/30374/task/87899/root (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87899 safe_time ceph txt unknown /proc/30374/task/87899/exe (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 87899 safe_time ceph NOFD /proc/30374/task/87899/fd (opendir: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88743 OpHistory ceph cwd unknown /proc/30374/task/88743/cwd (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88743 OpHistory ceph rtd unknown /proc/30374/task/88743/root (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88743 OpHistory ceph txt unknown /proc/30374/task/88743/exe (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88743 OpHistory ceph NOFD /proc/30374/task/88743/fd (opendir: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88744 mgr-fin ceph cwd unknown /proc/30374/task/88744/cwd (readlink: Permission denied) 2024-05-08T05:56:40.911 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88744 mgr-fin ceph rtd unknown /proc/30374/task/88744/root (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88744 mgr-fin ceph txt unknown /proc/30374/task/88744/exe (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88744 mgr-fin ceph NOFD /proc/30374/task/88744/fd (opendir: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88745 ms_dispat ceph cwd unknown /proc/30374/task/88745/cwd (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88745 ms_dispat ceph rtd unknown /proc/30374/task/88745/root (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88745 ms_dispat ceph txt unknown /proc/30374/task/88745/exe (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88745 ms_dispat ceph NOFD /proc/30374/task/88745/fd (opendir: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88746 ms_local ceph cwd unknown /proc/30374/task/88746/cwd (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88746 ms_local ceph rtd unknown /proc/30374/task/88746/root (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88746 ms_local ceph txt unknown /proc/30374/task/88746/exe (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88746 ms_local ceph NOFD /proc/30374/task/88746/fd (opendir: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88747 safe_time ceph cwd unknown /proc/30374/task/88747/cwd (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88747 safe_time ceph rtd unknown /proc/30374/task/88747/root (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88747 safe_time ceph txt unknown /proc/30374/task/88747/exe (readlink: Permission denied) 2024-05-08T05:56:40.912 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88747 safe_time ceph NOFD /proc/30374/task/88747/fd (opendir: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88748 cmdfin ceph cwd unknown /proc/30374/task/88748/cwd (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88748 cmdfin ceph rtd unknown /proc/30374/task/88748/root (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88748 cmdfin ceph txt unknown /proc/30374/task/88748/exe (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88748 cmdfin ceph NOFD /proc/30374/task/88748/fd (opendir: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88752 io_contex ceph cwd unknown /proc/30374/task/88752/cwd (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88752 io_contex ceph rtd unknown /proc/30374/task/88752/root (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88752 io_contex ceph txt unknown /proc/30374/task/88752/exe (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88752 io_contex ceph NOFD /proc/30374/task/88752/fd (opendir: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88753 io_contex ceph cwd unknown /proc/30374/task/88753/cwd (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88753 io_contex ceph rtd unknown /proc/30374/task/88753/root (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88753 io_contex ceph txt unknown /proc/30374/task/88753/exe (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88753 io_contex ceph NOFD /proc/30374/task/88753/fd (opendir: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88754 ceph_time ceph cwd unknown /proc/30374/task/88754/cwd (readlink: Permission denied) 2024-05-08T05:56:40.913 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88754 ceph_time ceph rtd unknown /proc/30374/task/88754/root (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88754 ceph_time ceph txt unknown /proc/30374/task/88754/exe (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88754 ceph_time ceph NOFD /proc/30374/task/88754/fd (opendir: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88755 ms_dispat ceph cwd unknown /proc/30374/task/88755/cwd (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88755 ms_dispat ceph rtd unknown /proc/30374/task/88755/root (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88755 ms_dispat ceph txt unknown /proc/30374/task/88755/exe (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88755 ms_dispat ceph NOFD /proc/30374/task/88755/fd (opendir: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88756 ms_local ceph cwd unknown /proc/30374/task/88756/cwd (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88756 ms_local ceph rtd unknown /proc/30374/task/88756/root (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88756 ms_local ceph txt unknown /proc/30374/task/88756/exe (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88756 ms_local ceph NOFD /proc/30374/task/88756/fd (opendir: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88757 safe_time ceph cwd unknown /proc/30374/task/88757/cwd (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88757 safe_time ceph rtd unknown /proc/30374/task/88757/root (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88757 safe_time ceph txt unknown /proc/30374/task/88757/exe (readlink: Permission denied) 2024-05-08T05:56:40.914 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88757 safe_time ceph NOFD /proc/30374/task/88757/fd (opendir: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88758 safe_time ceph cwd unknown /proc/30374/task/88758/cwd (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88758 safe_time ceph rtd unknown /proc/30374/task/88758/root (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88758 safe_time ceph txt unknown /proc/30374/task/88758/exe (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88758 safe_time ceph NOFD /proc/30374/task/88758/fd (opendir: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88759 balancer ceph cwd unknown /proc/30374/task/88759/cwd (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88759 balancer ceph rtd unknown /proc/30374/task/88759/root (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88759 balancer ceph txt unknown /proc/30374/task/88759/exe (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88759 balancer ceph NOFD /proc/30374/task/88759/fd (opendir: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88760 m-fin-bal ceph cwd unknown /proc/30374/task/88760/cwd (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88760 m-fin-bal ceph rtd unknown /proc/30374/task/88760/root (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88760 m-fin-bal ceph txt unknown /proc/30374/task/88760/exe (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88760 m-fin-bal ceph NOFD /proc/30374/task/88760/fd (opendir: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88761 mgr-fin ceph cwd unknown /proc/30374/task/88761/cwd (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88761 mgr-fin ceph rtd unknown /proc/30374/task/88761/root (readlink: Permission denied) 2024-05-08T05:56:40.915 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88761 mgr-fin ceph txt unknown /proc/30374/task/88761/exe (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88761 mgr-fin ceph NOFD /proc/30374/task/88761/fd (opendir: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88762 mgr-fin ceph cwd unknown /proc/30374/task/88762/cwd (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88762 mgr-fin ceph rtd unknown /proc/30374/task/88762/root (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88762 mgr-fin ceph txt unknown /proc/30374/task/88762/exe (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88762 mgr-fin ceph NOFD /proc/30374/task/88762/fd (opendir: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88763 mgr-fin ceph cwd unknown /proc/30374/task/88763/cwd (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88763 mgr-fin ceph rtd unknown /proc/30374/task/88763/root (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88763 mgr-fin ceph txt unknown /proc/30374/task/88763/exe (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88763 mgr-fin ceph NOFD /proc/30374/task/88763/fd (opendir: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88764 mgr-fin ceph cwd unknown /proc/30374/task/88764/cwd (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88764 mgr-fin ceph rtd unknown /proc/30374/task/88764/root (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88764 mgr-fin ceph txt unknown /proc/30374/task/88764/exe (readlink: Permission denied) 2024-05-08T05:56:40.916 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88764 mgr-fin ceph NOFD /proc/30374/task/88764/fd (opendir: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88765 mgr-fin ceph cwd unknown /proc/30374/task/88765/cwd (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88765 mgr-fin ceph rtd unknown /proc/30374/task/88765/root (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88765 mgr-fin ceph txt unknown /proc/30374/task/88765/exe (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88765 mgr-fin ceph NOFD /proc/30374/task/88765/fd (opendir: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88766 mgr-fin ceph cwd unknown /proc/30374/task/88766/cwd (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88766 mgr-fin ceph rtd unknown /proc/30374/task/88766/root (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88766 mgr-fin ceph txt unknown /proc/30374/task/88766/exe (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88766 mgr-fin ceph NOFD /proc/30374/task/88766/fd (opendir: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88767 mgr-fin ceph cwd unknown /proc/30374/task/88767/cwd (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88767 mgr-fin ceph rtd unknown /proc/30374/task/88767/root (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88767 mgr-fin ceph txt unknown /proc/30374/task/88767/exe (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88767 mgr-fin ceph NOFD /proc/30374/task/88767/fd (opendir: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88768 mgr-fin ceph cwd unknown /proc/30374/task/88768/cwd (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88768 mgr-fin ceph rtd unknown /proc/30374/task/88768/root (readlink: Permission denied) 2024-05-08T05:56:40.917 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88768 mgr-fin ceph txt unknown /proc/30374/task/88768/exe (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88768 mgr-fin ceph NOFD /proc/30374/task/88768/fd (opendir: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88769 mgr-fin ceph cwd unknown /proc/30374/task/88769/cwd (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88769 mgr-fin ceph rtd unknown /proc/30374/task/88769/root (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88769 mgr-fin ceph txt unknown /proc/30374/task/88769/exe (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88769 mgr-fin ceph NOFD /proc/30374/task/88769/fd (opendir: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88770 mgr-fin ceph cwd unknown /proc/30374/task/88770/cwd (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88770 mgr-fin ceph rtd unknown /proc/30374/task/88770/root (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88770 mgr-fin ceph txt unknown /proc/30374/task/88770/exe (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88770 mgr-fin ceph NOFD /proc/30374/task/88770/fd (opendir: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88771 mgr-fin ceph cwd unknown /proc/30374/task/88771/cwd (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88771 mgr-fin ceph rtd unknown /proc/30374/task/88771/root (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88771 mgr-fin ceph txt unknown /proc/30374/task/88771/exe (readlink: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88771 mgr-fin ceph NOFD /proc/30374/task/88771/fd (opendir: Permission denied) 2024-05-08T05:56:40.918 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88772 mgr-fin ceph cwd unknown /proc/30374/task/88772/cwd (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88772 mgr-fin ceph rtd unknown /proc/30374/task/88772/root (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88772 mgr-fin ceph txt unknown /proc/30374/task/88772/exe (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88772 mgr-fin ceph NOFD /proc/30374/task/88772/fd (opendir: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88773 mgr-fin ceph cwd unknown /proc/30374/task/88773/cwd (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88773 mgr-fin ceph rtd unknown /proc/30374/task/88773/root (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88773 mgr-fin ceph txt unknown /proc/30374/task/88773/exe (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88773 mgr-fin ceph NOFD /proc/30374/task/88773/fd (opendir: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88774 mgr-fin ceph cwd unknown /proc/30374/task/88774/cwd (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88774 mgr-fin ceph rtd unknown /proc/30374/task/88774/root (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88774 mgr-fin ceph txt unknown /proc/30374/task/88774/exe (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88774 mgr-fin ceph NOFD /proc/30374/task/88774/fd (opendir: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88775 mgr-fin ceph cwd unknown /proc/30374/task/88775/cwd (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88775 mgr-fin ceph rtd unknown /proc/30374/task/88775/root (readlink: Permission denied) 2024-05-08T05:56:40.919 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88775 mgr-fin ceph txt unknown /proc/30374/task/88775/exe (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88775 mgr-fin ceph NOFD /proc/30374/task/88775/fd (opendir: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88776 mgr-fin ceph cwd unknown /proc/30374/task/88776/cwd (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88776 mgr-fin ceph rtd unknown /proc/30374/task/88776/root (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88776 mgr-fin ceph txt unknown /proc/30374/task/88776/exe (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88776 mgr-fin ceph NOFD /proc/30374/task/88776/fd (opendir: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88777 cephadm ceph cwd unknown /proc/30374/task/88777/cwd (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88777 cephadm ceph rtd unknown /proc/30374/task/88777/root (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88777 cephadm ceph txt unknown /proc/30374/task/88777/exe (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88777 cephadm ceph NOFD /proc/30374/task/88777/fd (opendir: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88778 m-fin-cep ceph cwd unknown /proc/30374/task/88778/cwd (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88778 m-fin-cep ceph rtd unknown /proc/30374/task/88778/root (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88778 m-fin-cep ceph txt unknown /proc/30374/task/88778/exe (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88778 m-fin-cep ceph NOFD /proc/30374/task/88778/fd (opendir: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88779 mgr-fin ceph cwd unknown /proc/30374/task/88779/cwd (readlink: Permission denied) 2024-05-08T05:56:40.920 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88779 mgr-fin ceph rtd unknown /proc/30374/task/88779/root (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88779 mgr-fin ceph txt unknown /proc/30374/task/88779/exe (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88779 mgr-fin ceph NOFD /proc/30374/task/88779/fd (opendir: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88780 cephadm ceph cwd unknown /proc/30374/task/88780/cwd (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88780 cephadm ceph rtd unknown /proc/30374/task/88780/root (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88780 cephadm ceph txt unknown /proc/30374/task/88780/exe (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88780 cephadm ceph NOFD /proc/30374/task/88780/fd (opendir: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88781 mgr-fin ceph cwd unknown /proc/30374/task/88781/cwd (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88781 mgr-fin ceph rtd unknown /proc/30374/task/88781/root (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88781 mgr-fin ceph txt unknown /proc/30374/task/88781/exe (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88781 mgr-fin ceph NOFD /proc/30374/task/88781/fd (opendir: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88782 crash ceph cwd unknown /proc/30374/task/88782/cwd (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88782 crash ceph rtd unknown /proc/30374/task/88782/root (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88782 crash ceph txt unknown /proc/30374/task/88782/exe (readlink: Permission denied) 2024-05-08T05:56:40.921 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88782 crash ceph NOFD /proc/30374/task/88782/fd (opendir: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88783 m-fin-cra ceph cwd unknown /proc/30374/task/88783/cwd (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88783 m-fin-cra ceph rtd unknown /proc/30374/task/88783/root (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88783 m-fin-cra ceph txt unknown /proc/30374/task/88783/exe (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88783 m-fin-cra ceph NOFD /proc/30374/task/88783/fd (opendir: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88784 mgr-fin ceph cwd unknown /proc/30374/task/88784/cwd (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88784 mgr-fin ceph rtd unknown /proc/30374/task/88784/root (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88784 mgr-fin ceph txt unknown /proc/30374/task/88784/exe (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88784 mgr-fin ceph NOFD /proc/30374/task/88784/fd (opendir: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88785 mgr-fin ceph cwd unknown /proc/30374/task/88785/cwd (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88785 mgr-fin ceph rtd unknown /proc/30374/task/88785/root (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88785 mgr-fin ceph txt unknown /proc/30374/task/88785/exe (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88785 mgr-fin ceph NOFD /proc/30374/task/88785/fd (opendir: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88786 dashboard ceph cwd unknown /proc/30374/task/88786/cwd (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88786 dashboard ceph rtd unknown /proc/30374/task/88786/root (readlink: Permission denied) 2024-05-08T05:56:40.922 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88786 dashboard ceph txt unknown /proc/30374/task/88786/exe (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88786 dashboard ceph NOFD /proc/30374/task/88786/fd (opendir: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88787 mgr-fin ceph cwd unknown /proc/30374/task/88787/cwd (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88787 mgr-fin ceph rtd unknown /proc/30374/task/88787/root (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88787 mgr-fin ceph txt unknown /proc/30374/task/88787/exe (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88787 mgr-fin ceph NOFD /proc/30374/task/88787/fd (opendir: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88788 m-fin-das ceph cwd unknown /proc/30374/task/88788/cwd (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88788 m-fin-das ceph rtd unknown /proc/30374/task/88788/root (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88788 m-fin-das ceph txt unknown /proc/30374/task/88788/exe (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88788 m-fin-das ceph NOFD /proc/30374/task/88788/fd (opendir: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88789 mgr-fin ceph cwd unknown /proc/30374/task/88789/cwd (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88789 mgr-fin ceph rtd unknown /proc/30374/task/88789/root (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88789 mgr-fin ceph txt unknown /proc/30374/task/88789/exe (readlink: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88789 mgr-fin ceph NOFD /proc/30374/task/88789/fd (opendir: Permission denied) 2024-05-08T05:56:40.923 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88790 devicehea ceph cwd unknown /proc/30374/task/88790/cwd (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88790 devicehea ceph rtd unknown /proc/30374/task/88790/root (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88790 devicehea ceph txt unknown /proc/30374/task/88790/exe (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88790 devicehea ceph NOFD /proc/30374/task/88790/fd (opendir: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88791 mgr-fin ceph cwd unknown /proc/30374/task/88791/cwd (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88791 mgr-fin ceph rtd unknown /proc/30374/task/88791/root (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88791 mgr-fin ceph txt unknown /proc/30374/task/88791/exe (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88791 mgr-fin ceph NOFD /proc/30374/task/88791/fd (opendir: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88792 m-fin-dev ceph cwd unknown /proc/30374/task/88792/cwd (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88792 m-fin-dev ceph rtd unknown /proc/30374/task/88792/root (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88792 m-fin-dev ceph txt unknown /proc/30374/task/88792/exe (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88792 m-fin-dev ceph NOFD /proc/30374/task/88792/fd (opendir: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88793 mgr-fin ceph cwd unknown /proc/30374/task/88793/cwd (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88793 mgr-fin ceph rtd unknown /proc/30374/task/88793/root (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88793 mgr-fin ceph txt unknown /proc/30374/task/88793/exe (readlink: Permission denied) 2024-05-08T05:56:40.924 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88793 mgr-fin ceph NOFD /proc/30374/task/88793/fd (opendir: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88795 mgr-fin ceph cwd unknown /proc/30374/task/88795/cwd (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88795 mgr-fin ceph rtd unknown /proc/30374/task/88795/root (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88795 mgr-fin ceph txt unknown /proc/30374/task/88795/exe (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88795 mgr-fin ceph NOFD /proc/30374/task/88795/fd (opendir: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88796 m-fin-ios ceph cwd unknown /proc/30374/task/88796/cwd (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88796 m-fin-ios ceph rtd unknown /proc/30374/task/88796/root (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88796 m-fin-ios ceph txt unknown /proc/30374/task/88796/exe (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88796 m-fin-ios ceph NOFD /proc/30374/task/88796/fd (opendir: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88797 mgr-fin ceph cwd unknown /proc/30374/task/88797/cwd (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88797 mgr-fin ceph rtd unknown /proc/30374/task/88797/root (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88797 mgr-fin ceph txt unknown /proc/30374/task/88797/exe (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88797 mgr-fin ceph NOFD /proc/30374/task/88797/fd (opendir: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88798 mgr-fin ceph cwd unknown /proc/30374/task/88798/cwd (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88798 mgr-fin ceph rtd unknown /proc/30374/task/88798/root (readlink: Permission denied) 2024-05-08T05:56:40.925 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88798 mgr-fin ceph txt unknown /proc/30374/task/88798/exe (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88798 mgr-fin ceph NOFD /proc/30374/task/88798/fd (opendir: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88799 mgr-fin ceph cwd unknown /proc/30374/task/88799/cwd (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88799 mgr-fin ceph rtd unknown /proc/30374/task/88799/root (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88799 mgr-fin ceph txt unknown /proc/30374/task/88799/exe (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88799 mgr-fin ceph NOFD /proc/30374/task/88799/fd (opendir: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88801 m-fin-nfs ceph cwd unknown /proc/30374/task/88801/cwd (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88801 m-fin-nfs ceph rtd unknown /proc/30374/task/88801/root (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88801 m-fin-nfs ceph txt unknown /proc/30374/task/88801/exe (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88801 m-fin-nfs ceph NOFD /proc/30374/task/88801/fd (opendir: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88803 m-fin-orc ceph cwd unknown /proc/30374/task/88803/cwd (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88803 m-fin-orc ceph rtd unknown /proc/30374/task/88803/root (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88803 m-fin-orc ceph txt unknown /proc/30374/task/88803/exe (readlink: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88803 m-fin-orc ceph NOFD /proc/30374/task/88803/fd (opendir: Permission denied) 2024-05-08T05:56:40.926 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88804 pg_autosc ceph cwd unknown /proc/30374/task/88804/cwd (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88804 pg_autosc ceph rtd unknown /proc/30374/task/88804/root (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88804 pg_autosc ceph txt unknown /proc/30374/task/88804/exe (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88804 pg_autosc ceph NOFD /proc/30374/task/88804/fd (opendir: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88805 m-fin-pg_ ceph cwd unknown /proc/30374/task/88805/cwd (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88805 m-fin-pg_ ceph rtd unknown /proc/30374/task/88805/root (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88805 m-fin-pg_ ceph txt unknown /proc/30374/task/88805/exe (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88805 m-fin-pg_ ceph NOFD /proc/30374/task/88805/fd (opendir: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88806 progress ceph cwd unknown /proc/30374/task/88806/cwd (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88806 progress ceph rtd unknown /proc/30374/task/88806/root (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88806 progress ceph txt unknown /proc/30374/task/88806/exe (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88806 progress ceph NOFD /proc/30374/task/88806/fd (opendir: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88807 m-fin-pro ceph cwd unknown /proc/30374/task/88807/cwd (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88807 m-fin-pro ceph rtd unknown /proc/30374/task/88807/root (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88807 m-fin-pro ceph txt unknown /proc/30374/task/88807/exe (readlink: Permission denied) 2024-05-08T05:56:40.927 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88807 m-fin-pro ceph NOFD /proc/30374/task/88807/fd (opendir: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88811 io_contex ceph cwd unknown /proc/30374/task/88811/cwd (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88811 io_contex ceph rtd unknown /proc/30374/task/88811/root (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88811 io_contex ceph txt unknown /proc/30374/task/88811/exe (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88811 io_contex ceph NOFD /proc/30374/task/88811/fd (opendir: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88812 io_contex ceph cwd unknown /proc/30374/task/88812/cwd (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88812 io_contex ceph rtd unknown /proc/30374/task/88812/root (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88812 io_contex ceph txt unknown /proc/30374/task/88812/exe (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88812 io_contex ceph NOFD /proc/30374/task/88812/fd (opendir: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88813 ceph_time ceph cwd unknown /proc/30374/task/88813/cwd (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88813 ceph_time ceph rtd unknown /proc/30374/task/88813/root (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88813 ceph_time ceph txt unknown /proc/30374/task/88813/exe (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88813 ceph_time ceph NOFD /proc/30374/task/88813/fd (opendir: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88814 ms_dispat ceph cwd unknown /proc/30374/task/88814/cwd (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88814 ms_dispat ceph rtd unknown /proc/30374/task/88814/root (readlink: Permission denied) 2024-05-08T05:56:40.928 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88814 ms_dispat ceph txt unknown /proc/30374/task/88814/exe (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88814 ms_dispat ceph NOFD /proc/30374/task/88814/fd (opendir: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88815 ms_local ceph cwd unknown /proc/30374/task/88815/cwd (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88815 ms_local ceph rtd unknown /proc/30374/task/88815/root (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88815 ms_local ceph txt unknown /proc/30374/task/88815/exe (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88815 ms_local ceph NOFD /proc/30374/task/88815/fd (opendir: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88816 safe_time ceph cwd unknown /proc/30374/task/88816/cwd (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88816 safe_time ceph rtd unknown /proc/30374/task/88816/root (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88816 safe_time ceph txt unknown /proc/30374/task/88816/exe (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88816 safe_time ceph NOFD /proc/30374/task/88816/fd (opendir: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88817 safe_time ceph cwd unknown /proc/30374/task/88817/cwd (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88817 safe_time ceph rtd unknown /proc/30374/task/88817/root (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88817 safe_time ceph txt unknown /proc/30374/task/88817/exe (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88817 safe_time ceph NOFD /proc/30374/task/88817/fd (opendir: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88818 mgr-fin ceph cwd unknown /proc/30374/task/88818/cwd (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88818 mgr-fin ceph rtd unknown /proc/30374/task/88818/root (readlink: Permission denied) 2024-05-08T05:56:40.929 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88818 mgr-fin ceph txt unknown /proc/30374/task/88818/exe (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88818 mgr-fin ceph NOFD /proc/30374/task/88818/fd (opendir: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88820 m-fin-rbd ceph cwd unknown /proc/30374/task/88820/cwd (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88820 m-fin-rbd ceph rtd unknown /proc/30374/task/88820/root (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88820 m-fin-rbd ceph txt unknown /proc/30374/task/88820/exe (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88820 m-fin-rbd ceph NOFD /proc/30374/task/88820/fd (opendir: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88821 restful ceph cwd unknown /proc/30374/task/88821/cwd (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88821 restful ceph rtd unknown /proc/30374/task/88821/root (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88821 restful ceph txt unknown /proc/30374/task/88821/exe (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88821 restful ceph NOFD /proc/30374/task/88821/fd (opendir: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88822 m-fin-res ceph cwd unknown /proc/30374/task/88822/cwd (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88822 m-fin-res ceph rtd unknown /proc/30374/task/88822/root (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88822 m-fin-res ceph txt unknown /proc/30374/task/88822/exe (readlink: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88822 m-fin-res ceph NOFD /proc/30374/task/88822/fd (opendir: Permission denied) 2024-05-08T05:56:40.930 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88823 mgr-fin ceph cwd unknown /proc/30374/task/88823/cwd (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88823 mgr-fin ceph rtd unknown /proc/30374/task/88823/root (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88823 mgr-fin ceph txt unknown /proc/30374/task/88823/exe (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88823 mgr-fin ceph NOFD /proc/30374/task/88823/fd (opendir: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88824 mgr-fin ceph cwd unknown /proc/30374/task/88824/cwd (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88824 mgr-fin ceph rtd unknown /proc/30374/task/88824/root (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88824 mgr-fin ceph txt unknown /proc/30374/task/88824/exe (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88824 mgr-fin ceph NOFD /proc/30374/task/88824/fd (opendir: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88825 mgr-fin ceph cwd unknown /proc/30374/task/88825/cwd (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88825 mgr-fin ceph rtd unknown /proc/30374/task/88825/root (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88825 mgr-fin ceph txt unknown /proc/30374/task/88825/exe (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88825 mgr-fin ceph NOFD /proc/30374/task/88825/fd (opendir: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88826 mgr-fin ceph cwd unknown /proc/30374/task/88826/cwd (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88826 mgr-fin ceph rtd unknown /proc/30374/task/88826/root (readlink: Permission denied) 2024-05-08T05:56:40.931 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88826 mgr-fin ceph txt unknown /proc/30374/task/88826/exe (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88826 mgr-fin ceph NOFD /proc/30374/task/88826/fd (opendir: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88827 mgr-fin ceph cwd unknown /proc/30374/task/88827/cwd (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88827 mgr-fin ceph rtd unknown /proc/30374/task/88827/root (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88827 mgr-fin ceph txt unknown /proc/30374/task/88827/exe (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88827 mgr-fin ceph NOFD /proc/30374/task/88827/fd (opendir: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88828 mgr-fin ceph cwd unknown /proc/30374/task/88828/cwd (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88828 mgr-fin ceph rtd unknown /proc/30374/task/88828/root (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88828 mgr-fin ceph txt unknown /proc/30374/task/88828/exe (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88828 mgr-fin ceph NOFD /proc/30374/task/88828/fd (opendir: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88832 io_contex ceph cwd unknown /proc/30374/task/88832/cwd (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88832 io_contex ceph rtd unknown /proc/30374/task/88832/root (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88832 io_contex ceph txt unknown /proc/30374/task/88832/exe (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88832 io_contex ceph NOFD /proc/30374/task/88832/fd (opendir: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88833 io_contex ceph cwd unknown /proc/30374/task/88833/cwd (readlink: Permission denied) 2024-05-08T05:56:40.932 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88833 io_contex ceph rtd unknown /proc/30374/task/88833/root (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88833 io_contex ceph txt unknown /proc/30374/task/88833/exe (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88833 io_contex ceph NOFD /proc/30374/task/88833/fd (opendir: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88834 ceph_time ceph cwd unknown /proc/30374/task/88834/cwd (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88834 ceph_time ceph rtd unknown /proc/30374/task/88834/root (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88834 ceph_time ceph txt unknown /proc/30374/task/88834/exe (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88834 ceph_time ceph NOFD /proc/30374/task/88834/fd (opendir: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88835 ms_dispat ceph cwd unknown /proc/30374/task/88835/cwd (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88835 ms_dispat ceph rtd unknown /proc/30374/task/88835/root (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88835 ms_dispat ceph txt unknown /proc/30374/task/88835/exe (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88835 ms_dispat ceph NOFD /proc/30374/task/88835/fd (opendir: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88836 ms_local ceph cwd unknown /proc/30374/task/88836/cwd (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88836 ms_local ceph rtd unknown /proc/30374/task/88836/root (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88836 ms_local ceph txt unknown /proc/30374/task/88836/exe (readlink: Permission denied) 2024-05-08T05:56:40.933 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88836 ms_local ceph NOFD /proc/30374/task/88836/fd (opendir: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88837 safe_time ceph cwd unknown /proc/30374/task/88837/cwd (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88837 safe_time ceph rtd unknown /proc/30374/task/88837/root (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88837 safe_time ceph txt unknown /proc/30374/task/88837/exe (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88837 safe_time ceph NOFD /proc/30374/task/88837/fd (opendir: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88838 safe_time ceph cwd unknown /proc/30374/task/88838/cwd (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88838 safe_time ceph rtd unknown /proc/30374/task/88838/root (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88838 safe_time ceph txt unknown /proc/30374/task/88838/exe (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88838 safe_time ceph NOFD /proc/30374/task/88838/fd (opendir: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88842 m-fin-sna ceph cwd unknown /proc/30374/task/88842/cwd (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88842 m-fin-sna ceph rtd unknown /proc/30374/task/88842/root (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88842 m-fin-sna ceph txt unknown /proc/30374/task/88842/exe (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88842 m-fin-sna ceph NOFD /proc/30374/task/88842/fd (opendir: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88844 m-fin-sta ceph cwd unknown /proc/30374/task/88844/cwd (readlink: Permission denied) 2024-05-08T05:56:40.934 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88844 m-fin-sta ceph rtd unknown /proc/30374/task/88844/root (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88844 m-fin-sta ceph txt unknown /proc/30374/task/88844/exe (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88844 m-fin-sta ceph NOFD /proc/30374/task/88844/fd (opendir: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88845 telemetry ceph cwd unknown /proc/30374/task/88845/cwd (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88845 telemetry ceph rtd unknown /proc/30374/task/88845/root (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88845 telemetry ceph txt unknown /proc/30374/task/88845/exe (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88845 telemetry ceph NOFD /proc/30374/task/88845/fd (opendir: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88846 m-fin-tel ceph cwd unknown /proc/30374/task/88846/cwd (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88846 m-fin-tel ceph rtd unknown /proc/30374/task/88846/root (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88846 m-fin-tel ceph txt unknown /proc/30374/task/88846/exe (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88846 m-fin-tel ceph NOFD /proc/30374/task/88846/fd (opendir: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88847 mgr-fin ceph cwd unknown /proc/30374/task/88847/cwd (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88847 mgr-fin ceph rtd unknown /proc/30374/task/88847/root (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88847 mgr-fin ceph txt unknown /proc/30374/task/88847/exe (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88847 mgr-fin ceph NOFD /proc/30374/task/88847/fd (opendir: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88851 io_contex ceph cwd unknown /proc/30374/task/88851/cwd (readlink: Permission denied) 2024-05-08T05:56:40.935 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88851 io_contex ceph rtd unknown /proc/30374/task/88851/root (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88851 io_contex ceph txt unknown /proc/30374/task/88851/exe (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88851 io_contex ceph NOFD /proc/30374/task/88851/fd (opendir: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88852 io_contex ceph cwd unknown /proc/30374/task/88852/cwd (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88852 io_contex ceph rtd unknown /proc/30374/task/88852/root (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88852 io_contex ceph txt unknown /proc/30374/task/88852/exe (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88852 io_contex ceph NOFD /proc/30374/task/88852/fd (opendir: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88853 ceph_time ceph cwd unknown /proc/30374/task/88853/cwd (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88853 ceph_time ceph rtd unknown /proc/30374/task/88853/root (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88853 ceph_time ceph txt unknown /proc/30374/task/88853/exe (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88853 ceph_time ceph NOFD /proc/30374/task/88853/fd (opendir: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88854 ms_dispat ceph cwd unknown /proc/30374/task/88854/cwd (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88854 ms_dispat ceph rtd unknown /proc/30374/task/88854/root (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88854 ms_dispat ceph txt unknown /proc/30374/task/88854/exe (readlink: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88854 ms_dispat ceph NOFD /proc/30374/task/88854/fd (opendir: Permission denied) 2024-05-08T05:56:40.936 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88855 ms_local ceph cwd unknown /proc/30374/task/88855/cwd (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88855 ms_local ceph rtd unknown /proc/30374/task/88855/root (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88855 ms_local ceph txt unknown /proc/30374/task/88855/exe (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88855 ms_local ceph NOFD /proc/30374/task/88855/fd (opendir: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88856 safe_time ceph cwd unknown /proc/30374/task/88856/cwd (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88856 safe_time ceph rtd unknown /proc/30374/task/88856/root (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88856 safe_time ceph txt unknown /proc/30374/task/88856/exe (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88856 safe_time ceph NOFD /proc/30374/task/88856/fd (opendir: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88857 safe_time ceph cwd unknown /proc/30374/task/88857/cwd (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88857 safe_time ceph rtd unknown /proc/30374/task/88857/root (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88857 safe_time ceph txt unknown /proc/30374/task/88857/exe (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88857 safe_time ceph NOFD /proc/30374/task/88857/fd (opendir: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88858 mgr-fin ceph cwd unknown /proc/30374/task/88858/cwd (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88858 mgr-fin ceph rtd unknown /proc/30374/task/88858/root (readlink: Permission denied) 2024-05-08T05:56:40.937 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88858 mgr-fin ceph txt unknown /proc/30374/task/88858/exe (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88858 mgr-fin ceph NOFD /proc/30374/task/88858/fd (opendir: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88859 mgr-fin ceph cwd unknown /proc/30374/task/88859/cwd (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88859 mgr-fin ceph rtd unknown /proc/30374/task/88859/root (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88859 mgr-fin ceph txt unknown /proc/30374/task/88859/exe (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88859 mgr-fin ceph NOFD /proc/30374/task/88859/fd (opendir: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88860 mgr-fin ceph cwd unknown /proc/30374/task/88860/cwd (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88860 mgr-fin ceph rtd unknown /proc/30374/task/88860/root (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88860 mgr-fin ceph txt unknown /proc/30374/task/88860/exe (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88860 mgr-fin ceph NOFD /proc/30374/task/88860/fd (opendir: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88861 mgr-fin ceph cwd unknown /proc/30374/task/88861/cwd (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88861 mgr-fin ceph rtd unknown /proc/30374/task/88861/root (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88861 mgr-fin ceph txt unknown /proc/30374/task/88861/exe (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88861 mgr-fin ceph NOFD /proc/30374/task/88861/fd (opendir: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88862 mgr-fin ceph cwd unknown /proc/30374/task/88862/cwd (readlink: Permission denied) 2024-05-08T05:56:40.938 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88862 mgr-fin ceph rtd unknown /proc/30374/task/88862/root (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88862 mgr-fin ceph txt unknown /proc/30374/task/88862/exe (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88862 mgr-fin ceph NOFD /proc/30374/task/88862/fd (opendir: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88863 mgr-fin ceph cwd unknown /proc/30374/task/88863/cwd (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88863 mgr-fin ceph rtd unknown /proc/30374/task/88863/root (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88863 mgr-fin ceph txt unknown /proc/30374/task/88863/exe (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88863 mgr-fin ceph NOFD /proc/30374/task/88863/fd (opendir: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88864 mgr-fin ceph cwd unknown /proc/30374/task/88864/cwd (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88864 mgr-fin ceph rtd unknown /proc/30374/task/88864/root (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88864 mgr-fin ceph txt unknown /proc/30374/task/88864/exe (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88864 mgr-fin ceph NOFD /proc/30374/task/88864/fd (opendir: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88865 mgr-fin ceph cwd unknown /proc/30374/task/88865/cwd (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88865 mgr-fin ceph rtd unknown /proc/30374/task/88865/root (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88865 mgr-fin ceph txt unknown /proc/30374/task/88865/exe (readlink: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88865 mgr-fin ceph NOFD /proc/30374/task/88865/fd (opendir: Permission denied) 2024-05-08T05:56:40.939 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88866 mgr-fin ceph cwd unknown /proc/30374/task/88866/cwd (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88866 mgr-fin ceph rtd unknown /proc/30374/task/88866/root (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88866 mgr-fin ceph txt unknown /proc/30374/task/88866/exe (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88866 mgr-fin ceph NOFD /proc/30374/task/88866/fd (opendir: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88867 mgr-fin ceph cwd unknown /proc/30374/task/88867/cwd (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88867 mgr-fin ceph rtd unknown /proc/30374/task/88867/root (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88867 mgr-fin ceph txt unknown /proc/30374/task/88867/exe (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88867 mgr-fin ceph NOFD /proc/30374/task/88867/fd (opendir: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88868 mgr-fin ceph cwd unknown /proc/30374/task/88868/cwd (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88868 mgr-fin ceph rtd unknown /proc/30374/task/88868/root (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88868 mgr-fin ceph txt unknown /proc/30374/task/88868/exe (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88868 mgr-fin ceph NOFD /proc/30374/task/88868/fd (opendir: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88869 mgr-fin ceph cwd unknown /proc/30374/task/88869/cwd (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88869 mgr-fin ceph rtd unknown /proc/30374/task/88869/root (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88869 mgr-fin ceph txt unknown /proc/30374/task/88869/exe (readlink: Permission denied) 2024-05-08T05:56:40.940 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88869 mgr-fin ceph NOFD /proc/30374/task/88869/fd (opendir: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88871 m-fin-vol ceph cwd unknown /proc/30374/task/88871/cwd (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88871 m-fin-vol ceph rtd unknown /proc/30374/task/88871/root (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88871 m-fin-vol ceph txt unknown /proc/30374/task/88871/exe (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88871 m-fin-vol ceph NOFD /proc/30374/task/88871/fd (opendir: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88872 io_contex ceph cwd unknown /proc/30374/task/88872/cwd (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88872 io_contex ceph rtd unknown /proc/30374/task/88872/root (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88872 io_contex ceph txt unknown /proc/30374/task/88872/exe (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88872 io_contex ceph NOFD /proc/30374/task/88872/fd (opendir: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88873 io_contex ceph cwd unknown /proc/30374/task/88873/cwd (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88873 io_contex ceph rtd unknown /proc/30374/task/88873/root (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88873 io_contex ceph txt unknown /proc/30374/task/88873/exe (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88873 io_contex ceph NOFD /proc/30374/task/88873/fd (opendir: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88917 dashboard ceph cwd unknown /proc/30374/task/88917/cwd (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88917 dashboard ceph rtd unknown /proc/30374/task/88917/root (readlink: Permission denied) 2024-05-08T05:56:40.941 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88917 dashboard ceph txt unknown /proc/30374/task/88917/exe (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88917 dashboard ceph NOFD /proc/30374/task/88917/fd (opendir: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88918 dashboard ceph cwd unknown /proc/30374/task/88918/cwd (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88918 dashboard ceph rtd unknown /proc/30374/task/88918/root (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88918 dashboard ceph txt unknown /proc/30374/task/88918/exe (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88918 dashboard ceph NOFD /proc/30374/task/88918/fd (opendir: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88919 dashboard ceph cwd unknown /proc/30374/task/88919/cwd (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88919 dashboard ceph rtd unknown /proc/30374/task/88919/root (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88919 dashboard ceph txt unknown /proc/30374/task/88919/exe (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88919 dashboard ceph NOFD /proc/30374/task/88919/fd (opendir: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88920 dashboard ceph cwd unknown /proc/30374/task/88920/cwd (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88920 dashboard ceph rtd unknown /proc/30374/task/88920/root (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88920 dashboard ceph txt unknown /proc/30374/task/88920/exe (readlink: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88920 dashboard ceph NOFD /proc/30374/task/88920/fd (opendir: Permission denied) 2024-05-08T05:56:40.942 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88921 dashboard ceph cwd unknown /proc/30374/task/88921/cwd (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88921 dashboard ceph rtd unknown /proc/30374/task/88921/root (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88921 dashboard ceph txt unknown /proc/30374/task/88921/exe (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88921 dashboard ceph NOFD /proc/30374/task/88921/fd (opendir: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88922 dashboard ceph cwd unknown /proc/30374/task/88922/cwd (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88922 dashboard ceph rtd unknown /proc/30374/task/88922/root (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88922 dashboard ceph txt unknown /proc/30374/task/88922/exe (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88922 dashboard ceph NOFD /proc/30374/task/88922/fd (opendir: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88923 dashboard ceph cwd unknown /proc/30374/task/88923/cwd (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88923 dashboard ceph rtd unknown /proc/30374/task/88923/root (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88923 dashboard ceph txt unknown /proc/30374/task/88923/exe (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88923 dashboard ceph NOFD /proc/30374/task/88923/fd (opendir: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88924 dashboard ceph cwd unknown /proc/30374/task/88924/cwd (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88924 dashboard ceph rtd unknown /proc/30374/task/88924/root (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88924 dashboard ceph txt unknown /proc/30374/task/88924/exe (readlink: Permission denied) 2024-05-08T05:56:40.943 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88924 dashboard ceph NOFD /proc/30374/task/88924/fd (opendir: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88925 dashboard ceph cwd unknown /proc/30374/task/88925/cwd (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88925 dashboard ceph rtd unknown /proc/30374/task/88925/root (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88925 dashboard ceph txt unknown /proc/30374/task/88925/exe (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88925 dashboard ceph NOFD /proc/30374/task/88925/fd (opendir: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88926 dashboard ceph cwd unknown /proc/30374/task/88926/cwd (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88926 dashboard ceph rtd unknown /proc/30374/task/88926/root (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88926 dashboard ceph txt unknown /proc/30374/task/88926/exe (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88926 dashboard ceph NOFD /proc/30374/task/88926/fd (opendir: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88928 dashboard ceph cwd unknown /proc/30374/task/88928/cwd (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88928 dashboard ceph rtd unknown /proc/30374/task/88928/root (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88928 dashboard ceph txt unknown /proc/30374/task/88928/exe (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88928 dashboard ceph NOFD /proc/30374/task/88928/fd (opendir: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88978 dashboard ceph cwd unknown /proc/30374/task/88978/cwd (readlink: Permission denied) 2024-05-08T05:56:40.944 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88978 dashboard ceph rtd unknown /proc/30374/task/88978/root (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88978 dashboard ceph txt unknown /proc/30374/task/88978/exe (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88978 dashboard ceph NOFD /proc/30374/task/88978/fd (opendir: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88979 mgr-fin ceph cwd unknown /proc/30374/task/88979/cwd (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88979 mgr-fin ceph rtd unknown /proc/30374/task/88979/root (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88979 mgr-fin ceph txt unknown /proc/30374/task/88979/exe (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88979 mgr-fin ceph NOFD /proc/30374/task/88979/fd (opendir: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88980 mgr-fin ceph cwd unknown /proc/30374/task/88980/cwd (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88980 mgr-fin ceph rtd unknown /proc/30374/task/88980/root (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88980 mgr-fin ceph txt unknown /proc/30374/task/88980/exe (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88980 mgr-fin ceph NOFD /proc/30374/task/88980/fd (opendir: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88981 mgr-fin ceph cwd unknown /proc/30374/task/88981/cwd (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88981 mgr-fin ceph rtd unknown /proc/30374/task/88981/root (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88981 mgr-fin ceph txt unknown /proc/30374/task/88981/exe (readlink: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88981 mgr-fin ceph NOFD /proc/30374/task/88981/fd (opendir: Permission denied) 2024-05-08T05:56:40.945 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88982 mgr-fin ceph cwd unknown /proc/30374/task/88982/cwd (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88982 mgr-fin ceph rtd unknown /proc/30374/task/88982/root (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88982 mgr-fin ceph txt unknown /proc/30374/task/88982/exe (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88982 mgr-fin ceph NOFD /proc/30374/task/88982/fd (opendir: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88983 mgr-fin ceph cwd unknown /proc/30374/task/88983/cwd (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88983 mgr-fin ceph rtd unknown /proc/30374/task/88983/root (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88983 mgr-fin ceph txt unknown /proc/30374/task/88983/exe (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88983 mgr-fin ceph NOFD /proc/30374/task/88983/fd (opendir: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88984 mgr-fin ceph cwd unknown /proc/30374/task/88984/cwd (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88984 mgr-fin ceph rtd unknown /proc/30374/task/88984/root (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88984 mgr-fin ceph txt unknown /proc/30374/task/88984/exe (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88984 mgr-fin ceph NOFD /proc/30374/task/88984/fd (opendir: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88985 mgr-fin ceph cwd unknown /proc/30374/task/88985/cwd (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88985 mgr-fin ceph rtd unknown /proc/30374/task/88985/root (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88985 mgr-fin ceph txt unknown /proc/30374/task/88985/exe (readlink: Permission denied) 2024-05-08T05:56:40.946 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88985 mgr-fin ceph NOFD /proc/30374/task/88985/fd (opendir: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88986 mgr-fin ceph cwd unknown /proc/30374/task/88986/cwd (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88986 mgr-fin ceph rtd unknown /proc/30374/task/88986/root (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88986 mgr-fin ceph txt unknown /proc/30374/task/88986/exe (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88986 mgr-fin ceph NOFD /proc/30374/task/88986/fd (opendir: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88987 mgr-fin ceph cwd unknown /proc/30374/task/88987/cwd (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88987 mgr-fin ceph rtd unknown /proc/30374/task/88987/root (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88987 mgr-fin ceph txt unknown /proc/30374/task/88987/exe (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88987 mgr-fin ceph NOFD /proc/30374/task/88987/fd (opendir: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88988 mgr-fin ceph cwd unknown /proc/30374/task/88988/cwd (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88988 mgr-fin ceph rtd unknown /proc/30374/task/88988/root (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88988 mgr-fin ceph txt unknown /proc/30374/task/88988/exe (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88988 mgr-fin ceph NOFD /proc/30374/task/88988/fd (opendir: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88989 mgr-fin ceph cwd unknown /proc/30374/task/88989/cwd (readlink: Permission denied) 2024-05-08T05:56:40.947 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88989 mgr-fin ceph rtd unknown /proc/30374/task/88989/root (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88989 mgr-fin ceph txt unknown /proc/30374/task/88989/exe (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88989 mgr-fin ceph NOFD /proc/30374/task/88989/fd (opendir: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88991 mgr-fin ceph cwd unknown /proc/30374/task/88991/cwd (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88991 mgr-fin ceph rtd unknown /proc/30374/task/88991/root (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88991 mgr-fin ceph txt unknown /proc/30374/task/88991/exe (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88991 mgr-fin ceph NOFD /proc/30374/task/88991/fd (opendir: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88992 mgr-fin ceph cwd unknown /proc/30374/task/88992/cwd (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88992 mgr-fin ceph rtd unknown /proc/30374/task/88992/root (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88992 mgr-fin ceph txt unknown /proc/30374/task/88992/exe (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88992 mgr-fin ceph NOFD /proc/30374/task/88992/fd (opendir: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88993 mgr-fin ceph cwd unknown /proc/30374/task/88993/cwd (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88993 mgr-fin ceph rtd unknown /proc/30374/task/88993/root (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88993 mgr-fin ceph txt unknown /proc/30374/task/88993/exe (readlink: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88993 mgr-fin ceph NOFD /proc/30374/task/88993/fd (opendir: Permission denied) 2024-05-08T05:56:40.948 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88994 mgr-fin ceph cwd unknown /proc/30374/task/88994/cwd (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88994 mgr-fin ceph rtd unknown /proc/30374/task/88994/root (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88994 mgr-fin ceph txt unknown /proc/30374/task/88994/exe (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88994 mgr-fin ceph NOFD /proc/30374/task/88994/fd (opendir: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88995 mgr-fin ceph cwd unknown /proc/30374/task/88995/cwd (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88995 mgr-fin ceph rtd unknown /proc/30374/task/88995/root (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88995 mgr-fin ceph txt unknown /proc/30374/task/88995/exe (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88995 mgr-fin ceph NOFD /proc/30374/task/88995/fd (opendir: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88996 mgr-fin ceph cwd unknown /proc/30374/task/88996/cwd (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88996 mgr-fin ceph rtd unknown /proc/30374/task/88996/root (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88996 mgr-fin ceph txt unknown /proc/30374/task/88996/exe (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88996 mgr-fin ceph NOFD /proc/30374/task/88996/fd (opendir: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88997 mgr-fin ceph cwd unknown /proc/30374/task/88997/cwd (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88997 mgr-fin ceph rtd unknown /proc/30374/task/88997/root (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88997 mgr-fin ceph txt unknown /proc/30374/task/88997/exe (readlink: Permission denied) 2024-05-08T05:56:40.949 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88997 mgr-fin ceph NOFD /proc/30374/task/88997/fd (opendir: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88998 mgr-fin ceph cwd unknown /proc/30374/task/88998/cwd (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88998 mgr-fin ceph rtd unknown /proc/30374/task/88998/root (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88998 mgr-fin ceph txt unknown /proc/30374/task/88998/exe (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88998 mgr-fin ceph NOFD /proc/30374/task/88998/fd (opendir: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88999 mgr-fin ceph cwd unknown /proc/30374/task/88999/cwd (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88999 mgr-fin ceph rtd unknown /proc/30374/task/88999/root (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88999 mgr-fin ceph txt unknown /proc/30374/task/88999/exe (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 88999 mgr-fin ceph NOFD /proc/30374/task/88999/fd (opendir: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89000 mgr-fin ceph cwd unknown /proc/30374/task/89000/cwd (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89000 mgr-fin ceph rtd unknown /proc/30374/task/89000/root (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89000 mgr-fin ceph txt unknown /proc/30374/task/89000/exe (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89000 mgr-fin ceph NOFD /proc/30374/task/89000/fd (opendir: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89001 mgr-fin ceph cwd unknown /proc/30374/task/89001/cwd (readlink: Permission denied) 2024-05-08T05:56:40.950 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89001 mgr-fin ceph rtd unknown /proc/30374/task/89001/root (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89001 mgr-fin ceph txt unknown /proc/30374/task/89001/exe (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89001 mgr-fin ceph NOFD /proc/30374/task/89001/fd (opendir: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89314 devicehea ceph cwd unknown /proc/30374/task/89314/cwd (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89314 devicehea ceph rtd unknown /proc/30374/task/89314/root (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89314 devicehea ceph txt unknown /proc/30374/task/89314/exe (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89314 devicehea ceph NOFD /proc/30374/task/89314/fd (opendir: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89318 io_contex ceph cwd unknown /proc/30374/task/89318/cwd (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89318 io_contex ceph rtd unknown /proc/30374/task/89318/root (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89318 io_contex ceph txt unknown /proc/30374/task/89318/exe (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89318 io_contex ceph NOFD /proc/30374/task/89318/fd (opendir: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89319 io_contex ceph cwd unknown /proc/30374/task/89319/cwd (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89319 io_contex ceph rtd unknown /proc/30374/task/89319/root (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89319 io_contex ceph txt unknown /proc/30374/task/89319/exe (readlink: Permission denied) 2024-05-08T05:56:40.951 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89319 io_contex ceph NOFD /proc/30374/task/89319/fd (opendir: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89320 ceph_time ceph cwd unknown /proc/30374/task/89320/cwd (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89320 ceph_time ceph rtd unknown /proc/30374/task/89320/root (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89320 ceph_time ceph txt unknown /proc/30374/task/89320/exe (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89320 ceph_time ceph NOFD /proc/30374/task/89320/fd (opendir: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89321 ms_dispat ceph cwd unknown /proc/30374/task/89321/cwd (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89321 ms_dispat ceph rtd unknown /proc/30374/task/89321/root (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89321 ms_dispat ceph txt unknown /proc/30374/task/89321/exe (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89321 ms_dispat ceph NOFD /proc/30374/task/89321/fd (opendir: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89322 ms_local ceph cwd unknown /proc/30374/task/89322/cwd (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89322 ms_local ceph rtd unknown /proc/30374/task/89322/root (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89322 ms_local ceph txt unknown /proc/30374/task/89322/exe (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89322 ms_local ceph NOFD /proc/30374/task/89322/fd (opendir: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89323 safe_time ceph cwd unknown /proc/30374/task/89323/cwd (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89323 safe_time ceph rtd unknown /proc/30374/task/89323/root (readlink: Permission denied) 2024-05-08T05:56:40.952 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89323 safe_time ceph txt unknown /proc/30374/task/89323/exe (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89323 safe_time ceph NOFD /proc/30374/task/89323/fd (opendir: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89324 safe_time ceph cwd unknown /proc/30374/task/89324/cwd (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89324 safe_time ceph rtd unknown /proc/30374/task/89324/root (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89324 safe_time ceph txt unknown /proc/30374/task/89324/exe (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 89324 safe_time ceph NOFD /proc/30374/task/89324/fd (opendir: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91191 ceph_time ceph cwd unknown /proc/30374/task/91191/cwd (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91191 ceph_time ceph rtd unknown /proc/30374/task/91191/root (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91191 ceph_time ceph txt unknown /proc/30374/task/91191/exe (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91191 ceph_time ceph NOFD /proc/30374/task/91191/fd (opendir: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91192 fn_anonym ceph cwd unknown /proc/30374/task/91192/cwd (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91192 fn_anonym ceph rtd unknown /proc/30374/task/91192/root (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91192 fn_anonym ceph txt unknown /proc/30374/task/91192/exe (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91192 fn_anonym ceph NOFD /proc/30374/task/91192/fd (opendir: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91193 safe_time ceph cwd unknown /proc/30374/task/91193/cwd (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91193 safe_time ceph rtd unknown /proc/30374/task/91193/root (readlink: Permission denied) 2024-05-08T05:56:40.953 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91193 safe_time ceph txt unknown /proc/30374/task/91193/exe (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91193 safe_time ceph NOFD /proc/30374/task/91193/fd (opendir: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91194 fn_anonym ceph cwd unknown /proc/30374/task/91194/cwd (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91194 fn_anonym ceph rtd unknown /proc/30374/task/91194/root (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91194 fn_anonym ceph txt unknown /proc/30374/task/91194/exe (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91194 fn_anonym ceph NOFD /proc/30374/task/91194/fd (opendir: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91195 flusher ceph cwd unknown /proc/30374/task/91195/cwd (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91195 flusher ceph rtd unknown /proc/30374/task/91195/root (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91195 flusher ceph txt unknown /proc/30374/task/91195/exe (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91195 flusher ceph NOFD /proc/30374/task/91195/fd (opendir: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91196 ms_dispat ceph cwd unknown /proc/30374/task/91196/cwd (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91196 ms_dispat ceph rtd unknown /proc/30374/task/91196/root (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91196 ms_dispat ceph txt unknown /proc/30374/task/91196/exe (readlink: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91196 ms_dispat ceph NOFD /proc/30374/task/91196/fd (opendir: Permission denied) 2024-05-08T05:56:40.954 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91197 ms_local ceph cwd unknown /proc/30374/task/91197/cwd (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91197 ms_local ceph rtd unknown /proc/30374/task/91197/root (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91197 ms_local ceph txt unknown /proc/30374/task/91197/exe (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91197 ms_local ceph NOFD /proc/30374/task/91197/fd (opendir: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91198 safe_time ceph cwd unknown /proc/30374/task/91198/cwd (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91198 safe_time ceph rtd unknown /proc/30374/task/91198/root (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91198 safe_time ceph txt unknown /proc/30374/task/91198/exe (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91198 safe_time ceph NOFD /proc/30374/task/91198/fd (opendir: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91199 m-fin-sna ceph cwd unknown /proc/30374/task/91199/cwd (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91199 m-fin-sna ceph rtd unknown /proc/30374/task/91199/root (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91199 m-fin-sna ceph txt unknown /proc/30374/task/91199/exe (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 91199 m-fin-sna ceph NOFD /proc/30374/task/91199/fd (opendir: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 94315 m-fin-sna ceph cwd unknown /proc/30374/task/94315/cwd (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 94315 m-fin-sna ceph rtd unknown /proc/30374/task/94315/root (readlink: Permission denied) 2024-05-08T05:56:40.955 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 94315 m-fin-sna ceph txt unknown /proc/30374/task/94315/exe (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mgr 30374 94315 m-fin-sna ceph NOFD /proc/30374/task/94315/fd (opendir: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 30491 root cwd unknown /proc/30491/cwd (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 30491 root rtd unknown /proc/30491/root (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 30491 root txt unknown /proc/30491/exe (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 30491 root NOFD /proc/30491/fd (opendir: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 31468 root cwd unknown /proc/31468/cwd (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 31468 root rtd unknown /proc/31468/root (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 31468 root txt unknown /proc/31468/exe (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:systemd 31468 root NOFD /proc/31468/fd (opendir: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 31470 root cwd unknown /proc/31470/cwd (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 31470 root rtd unknown /proc/31470/root (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 31470 root txt unknown /proc/31470/exe (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:(sd-pam) 31470 root NOFD /proc/31470/fd (opendir: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 33647 root cwd unknown /proc/33647/cwd (readlink: Permission denied) 2024-05-08T05:56:40.956 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 33647 root rtd unknown /proc/33647/root (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 33647 root txt unknown /proc/33647/exe (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 33647 root NOFD /proc/33647/fd (opendir: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 44525 root cwd unknown /proc/44525/cwd (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 44525 root rtd unknown /proc/44525/root (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 44525 root txt unknown /proc/44525/exe (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 44525 root NOFD /proc/44525/fd (opendir: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 44527 root cwd unknown /proc/44527/cwd (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 44527 root rtd unknown /proc/44527/root (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 44527 root txt unknown /proc/44527/exe (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 44527 root NOFD /proc/44527/fd (opendir: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 ceph cwd unknown /proc/44529/cwd (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 ceph rtd unknown /proc/44529/root (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 ceph txt unknown /proc/44529/exe (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 ceph NOFD /proc/44529/fd (opendir: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44530 log ceph cwd unknown /proc/44529/task/44530/cwd (readlink: Permission denied) 2024-05-08T05:56:40.957 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44530 log ceph rtd unknown /proc/44529/task/44530/root (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44530 log ceph txt unknown /proc/44529/task/44530/exe (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44530 log ceph NOFD /proc/44529/task/44530/fd (opendir: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44532 msgr-work ceph cwd unknown /proc/44529/task/44532/cwd (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44532 msgr-work ceph rtd unknown /proc/44529/task/44532/root (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44532 msgr-work ceph txt unknown /proc/44529/task/44532/exe (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44532 msgr-work ceph NOFD /proc/44529/task/44532/fd (opendir: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44533 msgr-work ceph cwd unknown /proc/44529/task/44533/cwd (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44533 msgr-work ceph rtd unknown /proc/44529/task/44533/root (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44533 msgr-work ceph txt unknown /proc/44529/task/44533/exe (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44533 msgr-work ceph NOFD /proc/44529/task/44533/fd (opendir: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44534 msgr-work ceph cwd unknown /proc/44529/task/44534/cwd (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44534 msgr-work ceph rtd unknown /proc/44529/task/44534/root (readlink: Permission denied) 2024-05-08T05:56:40.958 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44534 msgr-work ceph txt unknown /proc/44529/task/44534/exe (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44534 msgr-work ceph NOFD /proc/44529/task/44534/fd (opendir: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44538 service ceph cwd unknown /proc/44529/task/44538/cwd (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44538 service ceph rtd unknown /proc/44529/task/44538/root (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44538 service ceph txt unknown /proc/44529/task/44538/exe (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44538 service ceph NOFD /proc/44529/task/44538/fd (opendir: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44539 admin_soc ceph cwd unknown /proc/44529/task/44539/cwd (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44539 admin_soc ceph rtd unknown /proc/44529/task/44539/root (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44539 admin_soc ceph txt unknown /proc/44529/task/44539/exe (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44539 admin_soc ceph NOFD /proc/44529/task/44539/fd (opendir: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44681 signal_ha ceph cwd unknown /proc/44529/task/44681/cwd (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44681 signal_ha ceph rtd unknown /proc/44529/task/44681/root (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44681 signal_ha ceph txt unknown /proc/44529/task/44681/exe (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44681 signal_ha ceph NOFD /proc/44529/task/44681/fd (opendir: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44682 io_contex ceph cwd unknown /proc/44529/task/44682/cwd (readlink: Permission denied) 2024-05-08T05:56:40.959 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44682 io_contex ceph rtd unknown /proc/44529/task/44682/root (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44682 io_contex ceph txt unknown /proc/44529/task/44682/exe (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44682 io_contex ceph NOFD /proc/44529/task/44682/fd (opendir: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44683 io_contex ceph cwd unknown /proc/44529/task/44683/cwd (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44683 io_contex ceph rtd unknown /proc/44529/task/44683/root (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44683 io_contex ceph txt unknown /proc/44529/task/44683/exe (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44683 io_contex ceph NOFD /proc/44529/task/44683/fd (opendir: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44798 OpHistory ceph cwd unknown /proc/44529/task/44798/cwd (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44798 OpHistory ceph rtd unknown /proc/44529/task/44798/root (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44798 OpHistory ceph txt unknown /proc/44529/task/44798/exe (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44798 OpHistory ceph NOFD /proc/44529/task/44798/fd (opendir: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44799 ceph_time ceph cwd unknown /proc/44529/task/44799/cwd (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44799 ceph_time ceph rtd unknown /proc/44529/task/44799/root (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44799 ceph_time ceph txt unknown /proc/44529/task/44799/exe (readlink: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44799 ceph_time ceph NOFD /proc/44529/task/44799/fd (opendir: Permission denied) 2024-05-08T05:56:40.960 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44822 ceph-osd ceph cwd unknown /proc/44529/task/44822/cwd (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44822 ceph-osd ceph rtd unknown /proc/44529/task/44822/root (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44822 ceph-osd ceph txt unknown /proc/44529/task/44822/exe (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44822 ceph-osd ceph NOFD /proc/44529/task/44822/fd (opendir: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44825 ceph-osd ceph cwd unknown /proc/44529/task/44825/cwd (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44825 ceph-osd ceph rtd unknown /proc/44529/task/44825/root (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44825 ceph-osd ceph txt unknown /proc/44529/task/44825/exe (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44825 ceph-osd ceph NOFD /proc/44529/task/44825/fd (opendir: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44860 ceph-osd ceph cwd unknown /proc/44529/task/44860/cwd (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44860 ceph-osd ceph rtd unknown /proc/44529/task/44860/root (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44860 ceph-osd ceph txt unknown /proc/44529/task/44860/exe (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44860 ceph-osd ceph NOFD /proc/44529/task/44860/fd (opendir: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44885 ceph-osd ceph cwd unknown /proc/44529/task/44885/cwd (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44885 ceph-osd ceph rtd unknown /proc/44529/task/44885/root (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44885 ceph-osd ceph txt unknown /proc/44529/task/44885/exe (readlink: Permission denied) 2024-05-08T05:56:40.961 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44885 ceph-osd ceph NOFD /proc/44529/task/44885/fd (opendir: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44905 ceph-osd ceph cwd unknown /proc/44529/task/44905/cwd (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44905 ceph-osd ceph rtd unknown /proc/44529/task/44905/root (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44905 ceph-osd ceph txt unknown /proc/44529/task/44905/exe (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44905 ceph-osd ceph NOFD /proc/44529/task/44905/fd (opendir: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44915 ceph-osd ceph cwd unknown /proc/44529/task/44915/cwd (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44915 ceph-osd ceph rtd unknown /proc/44529/task/44915/root (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44915 ceph-osd ceph txt unknown /proc/44529/task/44915/exe (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44915 ceph-osd ceph NOFD /proc/44529/task/44915/fd (opendir: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44940 ceph-osd ceph cwd unknown /proc/44529/task/44940/cwd (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44940 ceph-osd ceph rtd unknown /proc/44529/task/44940/root (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44940 ceph-osd ceph txt unknown /proc/44529/task/44940/exe (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44940 ceph-osd ceph NOFD /proc/44529/task/44940/fd (opendir: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44980 ceph-osd ceph cwd unknown /proc/44529/task/44980/cwd (readlink: Permission denied) 2024-05-08T05:56:40.962 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44980 ceph-osd ceph rtd unknown /proc/44529/task/44980/root (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44980 ceph-osd ceph txt unknown /proc/44529/task/44980/exe (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44980 ceph-osd ceph NOFD /proc/44529/task/44980/fd (opendir: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44981 ceph-osd ceph cwd unknown /proc/44529/task/44981/cwd (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44981 ceph-osd ceph rtd unknown /proc/44529/task/44981/root (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44981 ceph-osd ceph txt unknown /proc/44529/task/44981/exe (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44981 ceph-osd ceph NOFD /proc/44529/task/44981/fd (opendir: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44982 safe_time ceph cwd unknown /proc/44529/task/44982/cwd (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44982 safe_time ceph rtd unknown /proc/44529/task/44982/root (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44982 safe_time ceph txt unknown /proc/44529/task/44982/exe (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44982 safe_time ceph NOFD /proc/44529/task/44982/fd (opendir: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44983 safe_time ceph cwd unknown /proc/44529/task/44983/cwd (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44983 safe_time ceph rtd unknown /proc/44529/task/44983/root (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44983 safe_time ceph txt unknown /proc/44529/task/44983/exe (readlink: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44983 safe_time ceph NOFD /proc/44529/task/44983/fd (opendir: Permission denied) 2024-05-08T05:56:40.963 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44984 safe_time ceph cwd unknown /proc/44529/task/44984/cwd (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44984 safe_time ceph rtd unknown /proc/44529/task/44984/root (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44984 safe_time ceph txt unknown /proc/44529/task/44984/exe (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44984 safe_time ceph NOFD /proc/44529/task/44984/fd (opendir: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44985 safe_time ceph cwd unknown /proc/44529/task/44985/cwd (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44985 safe_time ceph rtd unknown /proc/44529/task/44985/root (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44985 safe_time ceph txt unknown /proc/44529/task/44985/exe (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44985 safe_time ceph NOFD /proc/44529/task/44985/fd (opendir: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44986 fn_anonym ceph cwd unknown /proc/44529/task/44986/cwd (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44986 fn_anonym ceph rtd unknown /proc/44529/task/44986/root (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44986 fn_anonym ceph txt unknown /proc/44529/task/44986/exe (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44986 fn_anonym ceph NOFD /proc/44529/task/44986/fd (opendir: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44990 rocksdb:l ceph cwd unknown /proc/44529/task/44990/cwd (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44990 rocksdb:l ceph rtd unknown /proc/44529/task/44990/root (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44990 rocksdb:l ceph txt unknown /proc/44529/task/44990/exe (readlink: Permission denied) 2024-05-08T05:56:40.964 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44990 rocksdb:l ceph NOFD /proc/44529/task/44990/fd (opendir: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44991 rocksdb:l ceph cwd unknown /proc/44529/task/44991/cwd (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44991 rocksdb:l ceph rtd unknown /proc/44529/task/44991/root (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44991 rocksdb:l ceph txt unknown /proc/44529/task/44991/exe (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44991 rocksdb:l ceph NOFD /proc/44529/task/44991/fd (opendir: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44992 rocksdb:l ceph cwd unknown /proc/44529/task/44992/cwd (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44992 rocksdb:l ceph rtd unknown /proc/44529/task/44992/root (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44992 rocksdb:l ceph txt unknown /proc/44529/task/44992/exe (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44992 rocksdb:l ceph NOFD /proc/44529/task/44992/fd (opendir: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44993 rocksdb:h ceph cwd unknown /proc/44529/task/44993/cwd (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44993 rocksdb:h ceph rtd unknown /proc/44529/task/44993/root (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44993 rocksdb:h ceph txt unknown /proc/44529/task/44993/exe (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 44993 rocksdb:h ceph NOFD /proc/44529/task/44993/fd (opendir: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45376 bstore_ai ceph cwd unknown /proc/44529/task/45376/cwd (readlink: Permission denied) 2024-05-08T05:56:40.965 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45376 bstore_ai ceph rtd unknown /proc/44529/task/45376/root (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45376 bstore_ai ceph txt unknown /proc/44529/task/45376/exe (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45376 bstore_ai ceph NOFD /proc/44529/task/45376/fd (opendir: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45558 bstore_ai ceph cwd unknown /proc/44529/task/45558/cwd (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45558 bstore_ai ceph rtd unknown /proc/44529/task/45558/root (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45558 bstore_ai ceph txt unknown /proc/44529/task/45558/exe (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45558 bstore_ai ceph NOFD /proc/44529/task/45558/fd (opendir: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45742 ceph-osd ceph cwd unknown /proc/44529/task/45742/cwd (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45742 ceph-osd ceph rtd unknown /proc/44529/task/45742/root (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45742 ceph-osd ceph txt unknown /proc/44529/task/45742/exe (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45742 ceph-osd ceph NOFD /proc/44529/task/45742/fd (opendir: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45743 cfin ceph cwd unknown /proc/44529/task/45743/cwd (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45743 cfin ceph rtd unknown /proc/44529/task/45743/root (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45743 cfin ceph txt unknown /proc/44529/task/45743/exe (readlink: Permission denied) 2024-05-08T05:56:40.966 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45743 cfin ceph NOFD /proc/44529/task/45743/fd (opendir: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45744 bstore_kv ceph cwd unknown /proc/44529/task/45744/cwd (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45744 bstore_kv ceph rtd unknown /proc/44529/task/45744/root (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45744 bstore_kv ceph txt unknown /proc/44529/task/45744/exe (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45744 bstore_kv ceph NOFD /proc/44529/task/45744/fd (opendir: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45745 bstore_kv ceph cwd unknown /proc/44529/task/45745/cwd (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45745 bstore_kv ceph rtd unknown /proc/44529/task/45745/root (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45745 bstore_kv ceph txt unknown /proc/44529/task/45745/exe (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45745 bstore_kv ceph NOFD /proc/44529/task/45745/fd (opendir: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45746 bstore_me ceph cwd unknown /proc/44529/task/45746/cwd (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45746 bstore_me ceph rtd unknown /proc/44529/task/45746/root (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45746 bstore_me ceph txt unknown /proc/44529/task/45746/exe (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45746 bstore_me ceph NOFD /proc/44529/task/45746/fd (opendir: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45747 ms_dispat ceph cwd unknown /proc/44529/task/45747/cwd (readlink: Permission denied) 2024-05-08T05:56:40.967 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45747 ms_dispat ceph rtd unknown /proc/44529/task/45747/root (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45747 ms_dispat ceph txt unknown /proc/44529/task/45747/exe (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45747 ms_dispat ceph NOFD /proc/44529/task/45747/fd (opendir: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45748 ms_local ceph cwd unknown /proc/44529/task/45748/cwd (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45748 ms_local ceph rtd unknown /proc/44529/task/45748/root (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45748 ms_local ceph txt unknown /proc/44529/task/45748/exe (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45748 ms_local ceph NOFD /proc/44529/task/45748/fd (opendir: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45749 safe_time ceph cwd unknown /proc/44529/task/45749/cwd (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45749 safe_time ceph rtd unknown /proc/44529/task/45749/root (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45749 safe_time ceph txt unknown /proc/44529/task/45749/exe (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45749 safe_time ceph NOFD /proc/44529/task/45749/fd (opendir: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45750 safe_time ceph cwd unknown /proc/44529/task/45750/cwd (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45750 safe_time ceph rtd unknown /proc/44529/task/45750/root (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45750 safe_time ceph txt unknown /proc/44529/task/45750/exe (readlink: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45750 safe_time ceph NOFD /proc/44529/task/45750/fd (opendir: Permission denied) 2024-05-08T05:56:40.968 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45751 ms_dispat ceph cwd unknown /proc/44529/task/45751/cwd (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45751 ms_dispat ceph rtd unknown /proc/44529/task/45751/root (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45751 ms_dispat ceph txt unknown /proc/44529/task/45751/exe (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45751 ms_dispat ceph NOFD /proc/44529/task/45751/fd (opendir: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45752 ms_local ceph cwd unknown /proc/44529/task/45752/cwd (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45752 ms_local ceph rtd unknown /proc/44529/task/45752/root (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45752 ms_local ceph txt unknown /proc/44529/task/45752/exe (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45752 ms_local ceph NOFD /proc/44529/task/45752/fd (opendir: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45753 ms_dispat ceph cwd unknown /proc/44529/task/45753/cwd (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45753 ms_dispat ceph rtd unknown /proc/44529/task/45753/root (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45753 ms_dispat ceph txt unknown /proc/44529/task/45753/exe (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45753 ms_dispat ceph NOFD /proc/44529/task/45753/fd (opendir: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45754 ms_local ceph cwd unknown /proc/44529/task/45754/cwd (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45754 ms_local ceph rtd unknown /proc/44529/task/45754/root (readlink: Permission denied) 2024-05-08T05:56:40.969 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45754 ms_local ceph txt unknown /proc/44529/task/45754/exe (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45754 ms_local ceph NOFD /proc/44529/task/45754/fd (opendir: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45755 ms_dispat ceph cwd unknown /proc/44529/task/45755/cwd (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45755 ms_dispat ceph rtd unknown /proc/44529/task/45755/root (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45755 ms_dispat ceph txt unknown /proc/44529/task/45755/exe (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45755 ms_dispat ceph NOFD /proc/44529/task/45755/fd (opendir: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45756 ms_local ceph cwd unknown /proc/44529/task/45756/cwd (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45756 ms_local ceph rtd unknown /proc/44529/task/45756/root (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45756 ms_local ceph txt unknown /proc/44529/task/45756/exe (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45756 ms_local ceph NOFD /proc/44529/task/45756/fd (opendir: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45757 ms_dispat ceph cwd unknown /proc/44529/task/45757/cwd (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45757 ms_dispat ceph rtd unknown /proc/44529/task/45757/root (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45757 ms_dispat ceph txt unknown /proc/44529/task/45757/exe (readlink: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45757 ms_dispat ceph NOFD /proc/44529/task/45757/fd (opendir: Permission denied) 2024-05-08T05:56:40.970 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45758 ms_local ceph cwd unknown /proc/44529/task/45758/cwd (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45758 ms_local ceph rtd unknown /proc/44529/task/45758/root (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45758 ms_local ceph txt unknown /proc/44529/task/45758/exe (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45758 ms_local ceph NOFD /proc/44529/task/45758/fd (opendir: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45759 ms_dispat ceph cwd unknown /proc/44529/task/45759/cwd (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45759 ms_dispat ceph rtd unknown /proc/44529/task/45759/root (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45759 ms_dispat ceph txt unknown /proc/44529/task/45759/exe (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45759 ms_dispat ceph NOFD /proc/44529/task/45759/fd (opendir: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45760 ms_local ceph cwd unknown /proc/44529/task/45760/cwd (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45760 ms_local ceph rtd unknown /proc/44529/task/45760/root (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45760 ms_local ceph txt unknown /proc/44529/task/45760/exe (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45760 ms_local ceph NOFD /proc/44529/task/45760/fd (opendir: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45761 ms_dispat ceph cwd unknown /proc/44529/task/45761/cwd (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45761 ms_dispat ceph rtd unknown /proc/44529/task/45761/root (readlink: Permission denied) 2024-05-08T05:56:40.971 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45761 ms_dispat ceph txt unknown /proc/44529/task/45761/exe (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45761 ms_dispat ceph NOFD /proc/44529/task/45761/fd (opendir: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45762 ms_local ceph cwd unknown /proc/44529/task/45762/cwd (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45762 ms_local ceph rtd unknown /proc/44529/task/45762/root (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45762 ms_local ceph txt unknown /proc/44529/task/45762/exe (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45762 ms_local ceph NOFD /proc/44529/task/45762/fd (opendir: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45763 fn_anonym ceph cwd unknown /proc/44529/task/45763/cwd (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45763 fn_anonym ceph rtd unknown /proc/44529/task/45763/root (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45763 fn_anonym ceph txt unknown /proc/44529/task/45763/exe (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45763 fn_anonym ceph NOFD /proc/44529/task/45763/fd (opendir: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45764 finisher ceph cwd unknown /proc/44529/task/45764/cwd (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45764 finisher ceph rtd unknown /proc/44529/task/45764/root (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45764 finisher ceph txt unknown /proc/44529/task/45764/exe (readlink: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45764 finisher ceph NOFD /proc/44529/task/45764/fd (opendir: Permission denied) 2024-05-08T05:56:40.972 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45765 safe_time ceph cwd unknown /proc/44529/task/45765/cwd (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45765 safe_time ceph rtd unknown /proc/44529/task/45765/root (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45765 safe_time ceph txt unknown /proc/44529/task/45765/exe (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45765 safe_time ceph NOFD /proc/44529/task/45765/fd (opendir: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45766 safe_time ceph cwd unknown /proc/44529/task/45766/cwd (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45766 safe_time ceph rtd unknown /proc/44529/task/45766/root (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45766 safe_time ceph txt unknown /proc/44529/task/45766/exe (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45766 safe_time ceph NOFD /proc/44529/task/45766/fd (opendir: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45767 ceph-osd ceph cwd unknown /proc/44529/task/45767/cwd (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45767 ceph-osd ceph rtd unknown /proc/44529/task/45767/root (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45767 ceph-osd ceph txt unknown /proc/44529/task/45767/exe (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45767 ceph-osd ceph NOFD /proc/44529/task/45767/fd (opendir: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45768 osd_srv_a ceph cwd unknown /proc/44529/task/45768/cwd (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45768 osd_srv_a ceph rtd unknown /proc/44529/task/45768/root (readlink: Permission denied) 2024-05-08T05:56:40.973 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45768 osd_srv_a ceph txt unknown /proc/44529/task/45768/exe (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45768 osd_srv_a ceph NOFD /proc/44529/task/45768/fd (opendir: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45769 tp_osd_tp ceph cwd unknown /proc/44529/task/45769/cwd (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45769 tp_osd_tp ceph rtd unknown /proc/44529/task/45769/root (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45769 tp_osd_tp ceph txt unknown /proc/44529/task/45769/exe (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45769 tp_osd_tp ceph NOFD /proc/44529/task/45769/fd (opendir: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45770 tp_osd_tp ceph cwd unknown /proc/44529/task/45770/cwd (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45770 tp_osd_tp ceph rtd unknown /proc/44529/task/45770/root (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45770 tp_osd_tp ceph txt unknown /proc/44529/task/45770/exe (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45770 tp_osd_tp ceph NOFD /proc/44529/task/45770/fd (opendir: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45771 tp_osd_tp ceph cwd unknown /proc/44529/task/45771/cwd (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45771 tp_osd_tp ceph rtd unknown /proc/44529/task/45771/root (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45771 tp_osd_tp ceph txt unknown /proc/44529/task/45771/exe (readlink: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45771 tp_osd_tp ceph NOFD /proc/44529/task/45771/fd (opendir: Permission denied) 2024-05-08T05:56:40.974 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45772 tp_osd_tp ceph cwd unknown /proc/44529/task/45772/cwd (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45772 tp_osd_tp ceph rtd unknown /proc/44529/task/45772/root (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45772 tp_osd_tp ceph txt unknown /proc/44529/task/45772/exe (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45772 tp_osd_tp ceph NOFD /proc/44529/task/45772/fd (opendir: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45773 tp_osd_tp ceph cwd unknown /proc/44529/task/45773/cwd (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45773 tp_osd_tp ceph rtd unknown /proc/44529/task/45773/root (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45773 tp_osd_tp ceph txt unknown /proc/44529/task/45773/exe (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45773 tp_osd_tp ceph NOFD /proc/44529/task/45773/fd (opendir: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45774 tp_osd_tp ceph cwd unknown /proc/44529/task/45774/cwd (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45774 tp_osd_tp ceph rtd unknown /proc/44529/task/45774/root (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45774 tp_osd_tp ceph txt unknown /proc/44529/task/45774/exe (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45774 tp_osd_tp ceph NOFD /proc/44529/task/45774/fd (opendir: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45775 tp_osd_tp ceph cwd unknown /proc/44529/task/45775/cwd (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45775 tp_osd_tp ceph rtd unknown /proc/44529/task/45775/root (readlink: Permission denied) 2024-05-08T05:56:40.975 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45775 tp_osd_tp ceph txt unknown /proc/44529/task/45775/exe (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45775 tp_osd_tp ceph NOFD /proc/44529/task/45775/fd (opendir: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45776 tp_osd_tp ceph cwd unknown /proc/44529/task/45776/cwd (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45776 tp_osd_tp ceph rtd unknown /proc/44529/task/45776/root (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45776 tp_osd_tp ceph txt unknown /proc/44529/task/45776/exe (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45776 tp_osd_tp ceph NOFD /proc/44529/task/45776/fd (opendir: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45777 tp_osd_tp ceph cwd unknown /proc/44529/task/45777/cwd (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45777 tp_osd_tp ceph rtd unknown /proc/44529/task/45777/root (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45777 tp_osd_tp ceph txt unknown /proc/44529/task/45777/exe (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45777 tp_osd_tp ceph NOFD /proc/44529/task/45777/fd (opendir: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45778 tp_osd_tp ceph cwd unknown /proc/44529/task/45778/cwd (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45778 tp_osd_tp ceph rtd unknown /proc/44529/task/45778/root (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45778 tp_osd_tp ceph txt unknown /proc/44529/task/45778/exe (readlink: Permission denied) 2024-05-08T05:56:40.976 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45778 tp_osd_tp ceph NOFD /proc/44529/task/45778/fd (opendir: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45779 tp_osd_tp ceph cwd unknown /proc/44529/task/45779/cwd (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45779 tp_osd_tp ceph rtd unknown /proc/44529/task/45779/root (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45779 tp_osd_tp ceph txt unknown /proc/44529/task/45779/exe (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45779 tp_osd_tp ceph NOFD /proc/44529/task/45779/fd (opendir: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45780 tp_osd_tp ceph cwd unknown /proc/44529/task/45780/cwd (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45780 tp_osd_tp ceph rtd unknown /proc/44529/task/45780/root (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45780 tp_osd_tp ceph txt unknown /proc/44529/task/45780/exe (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45780 tp_osd_tp ceph NOFD /proc/44529/task/45780/fd (opendir: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45781 tp_osd_tp ceph cwd unknown /proc/44529/task/45781/cwd (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45781 tp_osd_tp ceph rtd unknown /proc/44529/task/45781/root (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45781 tp_osd_tp ceph txt unknown /proc/44529/task/45781/exe (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45781 tp_osd_tp ceph NOFD /proc/44529/task/45781/fd (opendir: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45782 tp_osd_tp ceph cwd unknown /proc/44529/task/45782/cwd (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45782 tp_osd_tp ceph rtd unknown /proc/44529/task/45782/root (readlink: Permission denied) 2024-05-08T05:56:40.977 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45782 tp_osd_tp ceph txt unknown /proc/44529/task/45782/exe (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45782 tp_osd_tp ceph NOFD /proc/44529/task/45782/fd (opendir: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45783 tp_osd_tp ceph cwd unknown /proc/44529/task/45783/cwd (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45783 tp_osd_tp ceph rtd unknown /proc/44529/task/45783/root (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45783 tp_osd_tp ceph txt unknown /proc/44529/task/45783/exe (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45783 tp_osd_tp ceph NOFD /proc/44529/task/45783/fd (opendir: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45784 tp_osd_tp ceph cwd unknown /proc/44529/task/45784/cwd (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45784 tp_osd_tp ceph rtd unknown /proc/44529/task/45784/root (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45784 tp_osd_tp ceph txt unknown /proc/44529/task/45784/exe (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45784 tp_osd_tp ceph NOFD /proc/44529/task/45784/fd (opendir: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45785 osd_srv_h ceph cwd unknown /proc/44529/task/45785/cwd (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45785 osd_srv_h ceph rtd unknown /proc/44529/task/45785/root (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45785 osd_srv_h ceph txt unknown /proc/44529/task/45785/exe (readlink: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 44529 45785 osd_srv_h ceph NOFD /proc/44529/task/45785/fd (opendir: Permission denied) 2024-05-08T05:56:40.978 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 45786 root cwd unknown /proc/45786/cwd (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 45786 root rtd unknown /proc/45786/root (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 45786 root txt unknown /proc/45786/exe (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 45786 root NOFD /proc/45786/fd (opendir: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 45803 root cwd unknown /proc/45803/cwd (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 45803 root rtd unknown /proc/45803/root (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 45803 root txt unknown /proc/45803/exe (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 45803 root NOFD /proc/45803/fd (opendir: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 51457 root cwd unknown /proc/51457/cwd (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 51457 root rtd unknown /proc/51457/root (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 51457 root txt unknown /proc/51457/exe (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 51457 root NOFD /proc/51457/fd (opendir: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 51459 root cwd unknown /proc/51459/cwd (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 51459 root rtd unknown /proc/51459/root (readlink: Permission denied) 2024-05-08T05:56:40.979 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 51459 root txt unknown /proc/51459/exe (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 51459 root NOFD /proc/51459/fd (opendir: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 ceph cwd unknown /proc/51461/cwd (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 ceph rtd unknown /proc/51461/root (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 ceph txt unknown /proc/51461/exe (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 ceph NOFD /proc/51461/fd (opendir: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51462 log ceph cwd unknown /proc/51461/task/51462/cwd (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51462 log ceph rtd unknown /proc/51461/task/51462/root (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51462 log ceph txt unknown /proc/51461/task/51462/exe (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51462 log ceph NOFD /proc/51461/task/51462/fd (opendir: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51464 msgr-work ceph cwd unknown /proc/51461/task/51464/cwd (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51464 msgr-work ceph rtd unknown /proc/51461/task/51464/root (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51464 msgr-work ceph txt unknown /proc/51461/task/51464/exe (readlink: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51464 msgr-work ceph NOFD /proc/51461/task/51464/fd (opendir: Permission denied) 2024-05-08T05:56:40.980 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51465 msgr-work ceph cwd unknown /proc/51461/task/51465/cwd (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51465 msgr-work ceph rtd unknown /proc/51461/task/51465/root (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51465 msgr-work ceph txt unknown /proc/51461/task/51465/exe (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51465 msgr-work ceph NOFD /proc/51461/task/51465/fd (opendir: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51466 msgr-work ceph cwd unknown /proc/51461/task/51466/cwd (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51466 msgr-work ceph rtd unknown /proc/51461/task/51466/root (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51466 msgr-work ceph txt unknown /proc/51461/task/51466/exe (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51466 msgr-work ceph NOFD /proc/51461/task/51466/fd (opendir: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51470 service ceph cwd unknown /proc/51461/task/51470/cwd (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51470 service ceph rtd unknown /proc/51461/task/51470/root (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51470 service ceph txt unknown /proc/51461/task/51470/exe (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51470 service ceph NOFD /proc/51461/task/51470/fd (opendir: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51471 admin_soc ceph cwd unknown /proc/51461/task/51471/cwd (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51471 admin_soc ceph rtd unknown /proc/51461/task/51471/root (readlink: Permission denied) 2024-05-08T05:56:40.981 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51471 admin_soc ceph txt unknown /proc/51461/task/51471/exe (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51471 admin_soc ceph NOFD /proc/51461/task/51471/fd (opendir: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51614 signal_ha ceph cwd unknown /proc/51461/task/51614/cwd (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51614 signal_ha ceph rtd unknown /proc/51461/task/51614/root (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51614 signal_ha ceph txt unknown /proc/51461/task/51614/exe (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51614 signal_ha ceph NOFD /proc/51461/task/51614/fd (opendir: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51615 io_contex ceph cwd unknown /proc/51461/task/51615/cwd (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51615 io_contex ceph rtd unknown /proc/51461/task/51615/root (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51615 io_contex ceph txt unknown /proc/51461/task/51615/exe (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51615 io_contex ceph NOFD /proc/51461/task/51615/fd (opendir: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51616 io_contex ceph cwd unknown /proc/51461/task/51616/cwd (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51616 io_contex ceph rtd unknown /proc/51461/task/51616/root (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51616 io_contex ceph txt unknown /proc/51461/task/51616/exe (readlink: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51616 io_contex ceph NOFD /proc/51461/task/51616/fd (opendir: Permission denied) 2024-05-08T05:56:40.982 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51725 OpHistory ceph cwd unknown /proc/51461/task/51725/cwd (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51725 OpHistory ceph rtd unknown /proc/51461/task/51725/root (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51725 OpHistory ceph txt unknown /proc/51461/task/51725/exe (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51725 OpHistory ceph NOFD /proc/51461/task/51725/fd (opendir: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51726 ceph_time ceph cwd unknown /proc/51461/task/51726/cwd (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51726 ceph_time ceph rtd unknown /proc/51461/task/51726/root (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51726 ceph_time ceph txt unknown /proc/51461/task/51726/exe (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51726 ceph_time ceph NOFD /proc/51461/task/51726/fd (opendir: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51759 ceph-osd ceph cwd unknown /proc/51461/task/51759/cwd (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51759 ceph-osd ceph rtd unknown /proc/51461/task/51759/root (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51759 ceph-osd ceph txt unknown /proc/51461/task/51759/exe (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51759 ceph-osd ceph NOFD /proc/51461/task/51759/fd (opendir: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51764 ceph-osd ceph cwd unknown /proc/51461/task/51764/cwd (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51764 ceph-osd ceph rtd unknown /proc/51461/task/51764/root (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51764 ceph-osd ceph txt unknown /proc/51461/task/51764/exe (readlink: Permission denied) 2024-05-08T05:56:40.983 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51764 ceph-osd ceph NOFD /proc/51461/task/51764/fd (opendir: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51771 ceph-osd ceph cwd unknown /proc/51461/task/51771/cwd (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51771 ceph-osd ceph rtd unknown /proc/51461/task/51771/root (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51771 ceph-osd ceph txt unknown /proc/51461/task/51771/exe (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51771 ceph-osd ceph NOFD /proc/51461/task/51771/fd (opendir: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51776 ceph-osd ceph cwd unknown /proc/51461/task/51776/cwd (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51776 ceph-osd ceph rtd unknown /proc/51461/task/51776/root (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51776 ceph-osd ceph txt unknown /proc/51461/task/51776/exe (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51776 ceph-osd ceph NOFD /proc/51461/task/51776/fd (opendir: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51814 ceph-osd ceph cwd unknown /proc/51461/task/51814/cwd (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51814 ceph-osd ceph rtd unknown /proc/51461/task/51814/root (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51814 ceph-osd ceph txt unknown /proc/51461/task/51814/exe (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51814 ceph-osd ceph NOFD /proc/51461/task/51814/fd (opendir: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51830 ceph-osd ceph cwd unknown /proc/51461/task/51830/cwd (readlink: Permission denied) 2024-05-08T05:56:40.984 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51830 ceph-osd ceph rtd unknown /proc/51461/task/51830/root (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51830 ceph-osd ceph txt unknown /proc/51461/task/51830/exe (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51830 ceph-osd ceph NOFD /proc/51461/task/51830/fd (opendir: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51852 ceph-osd ceph cwd unknown /proc/51461/task/51852/cwd (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51852 ceph-osd ceph rtd unknown /proc/51461/task/51852/root (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51852 ceph-osd ceph txt unknown /proc/51461/task/51852/exe (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51852 ceph-osd ceph NOFD /proc/51461/task/51852/fd (opendir: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51896 ceph-osd ceph cwd unknown /proc/51461/task/51896/cwd (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51896 ceph-osd ceph rtd unknown /proc/51461/task/51896/root (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51896 ceph-osd ceph txt unknown /proc/51461/task/51896/exe (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51896 ceph-osd ceph NOFD /proc/51461/task/51896/fd (opendir: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51898 ceph-osd ceph cwd unknown /proc/51461/task/51898/cwd (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51898 ceph-osd ceph rtd unknown /proc/51461/task/51898/root (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51898 ceph-osd ceph txt unknown /proc/51461/task/51898/exe (readlink: Permission denied) 2024-05-08T05:56:40.985 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51898 ceph-osd ceph NOFD /proc/51461/task/51898/fd (opendir: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51899 safe_time ceph cwd unknown /proc/51461/task/51899/cwd (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51899 safe_time ceph rtd unknown /proc/51461/task/51899/root (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51899 safe_time ceph txt unknown /proc/51461/task/51899/exe (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51899 safe_time ceph NOFD /proc/51461/task/51899/fd (opendir: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51900 safe_time ceph cwd unknown /proc/51461/task/51900/cwd (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51900 safe_time ceph rtd unknown /proc/51461/task/51900/root (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51900 safe_time ceph txt unknown /proc/51461/task/51900/exe (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51900 safe_time ceph NOFD /proc/51461/task/51900/fd (opendir: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51901 safe_time ceph cwd unknown /proc/51461/task/51901/cwd (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51901 safe_time ceph rtd unknown /proc/51461/task/51901/root (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51901 safe_time ceph txt unknown /proc/51461/task/51901/exe (readlink: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51901 safe_time ceph NOFD /proc/51461/task/51901/fd (opendir: Permission denied) 2024-05-08T05:56:40.986 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51902 safe_time ceph cwd unknown /proc/51461/task/51902/cwd (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51902 safe_time ceph rtd unknown /proc/51461/task/51902/root (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51902 safe_time ceph txt unknown /proc/51461/task/51902/exe (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51902 safe_time ceph NOFD /proc/51461/task/51902/fd (opendir: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51903 fn_anonym ceph cwd unknown /proc/51461/task/51903/cwd (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51903 fn_anonym ceph rtd unknown /proc/51461/task/51903/root (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51903 fn_anonym ceph txt unknown /proc/51461/task/51903/exe (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51903 fn_anonym ceph NOFD /proc/51461/task/51903/fd (opendir: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51907 rocksdb:l ceph cwd unknown /proc/51461/task/51907/cwd (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51907 rocksdb:l ceph rtd unknown /proc/51461/task/51907/root (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51907 rocksdb:l ceph txt unknown /proc/51461/task/51907/exe (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51907 rocksdb:l ceph NOFD /proc/51461/task/51907/fd (opendir: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51908 rocksdb:l ceph cwd unknown /proc/51461/task/51908/cwd (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51908 rocksdb:l ceph rtd unknown /proc/51461/task/51908/root (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51908 rocksdb:l ceph txt unknown /proc/51461/task/51908/exe (readlink: Permission denied) 2024-05-08T05:56:40.987 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51908 rocksdb:l ceph NOFD /proc/51461/task/51908/fd (opendir: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51909 rocksdb:l ceph cwd unknown /proc/51461/task/51909/cwd (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51909 rocksdb:l ceph rtd unknown /proc/51461/task/51909/root (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51909 rocksdb:l ceph txt unknown /proc/51461/task/51909/exe (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51909 rocksdb:l ceph NOFD /proc/51461/task/51909/fd (opendir: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51910 rocksdb:h ceph cwd unknown /proc/51461/task/51910/cwd (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51910 rocksdb:h ceph rtd unknown /proc/51461/task/51910/root (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51910 rocksdb:h ceph txt unknown /proc/51461/task/51910/exe (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 51910 rocksdb:h ceph NOFD /proc/51461/task/51910/fd (opendir: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52306 bstore_ai ceph cwd unknown /proc/51461/task/52306/cwd (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52306 bstore_ai ceph rtd unknown /proc/51461/task/52306/root (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52306 bstore_ai ceph txt unknown /proc/51461/task/52306/exe (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52306 bstore_ai ceph NOFD /proc/51461/task/52306/fd (opendir: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52528 bstore_ai ceph cwd unknown /proc/51461/task/52528/cwd (readlink: Permission denied) 2024-05-08T05:56:40.988 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52528 bstore_ai ceph rtd unknown /proc/51461/task/52528/root (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52528 bstore_ai ceph txt unknown /proc/51461/task/52528/exe (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52528 bstore_ai ceph NOFD /proc/51461/task/52528/fd (opendir: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52711 ceph-osd ceph cwd unknown /proc/51461/task/52711/cwd (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52711 ceph-osd ceph rtd unknown /proc/51461/task/52711/root (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52711 ceph-osd ceph txt unknown /proc/51461/task/52711/exe (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52711 ceph-osd ceph NOFD /proc/51461/task/52711/fd (opendir: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52712 cfin ceph cwd unknown /proc/51461/task/52712/cwd (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52712 cfin ceph rtd unknown /proc/51461/task/52712/root (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52712 cfin ceph txt unknown /proc/51461/task/52712/exe (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52712 cfin ceph NOFD /proc/51461/task/52712/fd (opendir: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52713 bstore_kv ceph cwd unknown /proc/51461/task/52713/cwd (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52713 bstore_kv ceph rtd unknown /proc/51461/task/52713/root (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52713 bstore_kv ceph txt unknown /proc/51461/task/52713/exe (readlink: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52713 bstore_kv ceph NOFD /proc/51461/task/52713/fd (opendir: Permission denied) 2024-05-08T05:56:40.989 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52714 bstore_kv ceph cwd unknown /proc/51461/task/52714/cwd (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52714 bstore_kv ceph rtd unknown /proc/51461/task/52714/root (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52714 bstore_kv ceph txt unknown /proc/51461/task/52714/exe (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52714 bstore_kv ceph NOFD /proc/51461/task/52714/fd (opendir: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52715 bstore_me ceph cwd unknown /proc/51461/task/52715/cwd (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52715 bstore_me ceph rtd unknown /proc/51461/task/52715/root (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52715 bstore_me ceph txt unknown /proc/51461/task/52715/exe (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52715 bstore_me ceph NOFD /proc/51461/task/52715/fd (opendir: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52716 ms_dispat ceph cwd unknown /proc/51461/task/52716/cwd (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52716 ms_dispat ceph rtd unknown /proc/51461/task/52716/root (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52716 ms_dispat ceph txt unknown /proc/51461/task/52716/exe (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52716 ms_dispat ceph NOFD /proc/51461/task/52716/fd (opendir: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52717 ms_local ceph cwd unknown /proc/51461/task/52717/cwd (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52717 ms_local ceph rtd unknown /proc/51461/task/52717/root (readlink: Permission denied) 2024-05-08T05:56:40.990 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52717 ms_local ceph txt unknown /proc/51461/task/52717/exe (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52717 ms_local ceph NOFD /proc/51461/task/52717/fd (opendir: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52718 safe_time ceph cwd unknown /proc/51461/task/52718/cwd (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52718 safe_time ceph rtd unknown /proc/51461/task/52718/root (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52718 safe_time ceph txt unknown /proc/51461/task/52718/exe (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52718 safe_time ceph NOFD /proc/51461/task/52718/fd (opendir: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52719 safe_time ceph cwd unknown /proc/51461/task/52719/cwd (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52719 safe_time ceph rtd unknown /proc/51461/task/52719/root (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52719 safe_time ceph txt unknown /proc/51461/task/52719/exe (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52719 safe_time ceph NOFD /proc/51461/task/52719/fd (opendir: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52720 ms_dispat ceph cwd unknown /proc/51461/task/52720/cwd (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52720 ms_dispat ceph rtd unknown /proc/51461/task/52720/root (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52720 ms_dispat ceph txt unknown /proc/51461/task/52720/exe (readlink: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52720 ms_dispat ceph NOFD /proc/51461/task/52720/fd (opendir: Permission denied) 2024-05-08T05:56:40.991 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52721 ms_local ceph cwd unknown /proc/51461/task/52721/cwd (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52721 ms_local ceph rtd unknown /proc/51461/task/52721/root (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52721 ms_local ceph txt unknown /proc/51461/task/52721/exe (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52721 ms_local ceph NOFD /proc/51461/task/52721/fd (opendir: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52722 ms_dispat ceph cwd unknown /proc/51461/task/52722/cwd (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52722 ms_dispat ceph rtd unknown /proc/51461/task/52722/root (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52722 ms_dispat ceph txt unknown /proc/51461/task/52722/exe (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52722 ms_dispat ceph NOFD /proc/51461/task/52722/fd (opendir: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52723 ms_local ceph cwd unknown /proc/51461/task/52723/cwd (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52723 ms_local ceph rtd unknown /proc/51461/task/52723/root (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52723 ms_local ceph txt unknown /proc/51461/task/52723/exe (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52723 ms_local ceph NOFD /proc/51461/task/52723/fd (opendir: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52724 ms_dispat ceph cwd unknown /proc/51461/task/52724/cwd (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52724 ms_dispat ceph rtd unknown /proc/51461/task/52724/root (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52724 ms_dispat ceph txt unknown /proc/51461/task/52724/exe (readlink: Permission denied) 2024-05-08T05:56:40.992 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52724 ms_dispat ceph NOFD /proc/51461/task/52724/fd (opendir: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52725 ms_local ceph cwd unknown /proc/51461/task/52725/cwd (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52725 ms_local ceph rtd unknown /proc/51461/task/52725/root (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52725 ms_local ceph txt unknown /proc/51461/task/52725/exe (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52725 ms_local ceph NOFD /proc/51461/task/52725/fd (opendir: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52726 ms_dispat ceph cwd unknown /proc/51461/task/52726/cwd (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52726 ms_dispat ceph rtd unknown /proc/51461/task/52726/root (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52726 ms_dispat ceph txt unknown /proc/51461/task/52726/exe (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52726 ms_dispat ceph NOFD /proc/51461/task/52726/fd (opendir: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52727 ms_local ceph cwd unknown /proc/51461/task/52727/cwd (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52727 ms_local ceph rtd unknown /proc/51461/task/52727/root (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52727 ms_local ceph txt unknown /proc/51461/task/52727/exe (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52727 ms_local ceph NOFD /proc/51461/task/52727/fd (opendir: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52728 ms_dispat ceph cwd unknown /proc/51461/task/52728/cwd (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52728 ms_dispat ceph rtd unknown /proc/51461/task/52728/root (readlink: Permission denied) 2024-05-08T05:56:40.993 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52728 ms_dispat ceph txt unknown /proc/51461/task/52728/exe (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52728 ms_dispat ceph NOFD /proc/51461/task/52728/fd (opendir: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52729 ms_local ceph cwd unknown /proc/51461/task/52729/cwd (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52729 ms_local ceph rtd unknown /proc/51461/task/52729/root (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52729 ms_local ceph txt unknown /proc/51461/task/52729/exe (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52729 ms_local ceph NOFD /proc/51461/task/52729/fd (opendir: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52730 ms_dispat ceph cwd unknown /proc/51461/task/52730/cwd (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52730 ms_dispat ceph rtd unknown /proc/51461/task/52730/root (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52730 ms_dispat ceph txt unknown /proc/51461/task/52730/exe (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52730 ms_dispat ceph NOFD /proc/51461/task/52730/fd (opendir: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52731 ms_local ceph cwd unknown /proc/51461/task/52731/cwd (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52731 ms_local ceph rtd unknown /proc/51461/task/52731/root (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52731 ms_local ceph txt unknown /proc/51461/task/52731/exe (readlink: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52731 ms_local ceph NOFD /proc/51461/task/52731/fd (opendir: Permission denied) 2024-05-08T05:56:40.994 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52732 fn_anonym ceph cwd unknown /proc/51461/task/52732/cwd (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52732 fn_anonym ceph rtd unknown /proc/51461/task/52732/root (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52732 fn_anonym ceph txt unknown /proc/51461/task/52732/exe (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52732 fn_anonym ceph NOFD /proc/51461/task/52732/fd (opendir: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52733 finisher ceph cwd unknown /proc/51461/task/52733/cwd (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52733 finisher ceph rtd unknown /proc/51461/task/52733/root (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52733 finisher ceph txt unknown /proc/51461/task/52733/exe (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52733 finisher ceph NOFD /proc/51461/task/52733/fd (opendir: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52734 safe_time ceph cwd unknown /proc/51461/task/52734/cwd (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52734 safe_time ceph rtd unknown /proc/51461/task/52734/root (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52734 safe_time ceph txt unknown /proc/51461/task/52734/exe (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52734 safe_time ceph NOFD /proc/51461/task/52734/fd (opendir: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52735 safe_time ceph cwd unknown /proc/51461/task/52735/cwd (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52735 safe_time ceph rtd unknown /proc/51461/task/52735/root (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52735 safe_time ceph txt unknown /proc/51461/task/52735/exe (readlink: Permission denied) 2024-05-08T05:56:40.995 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52735 safe_time ceph NOFD /proc/51461/task/52735/fd (opendir: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52736 ceph-osd ceph cwd unknown /proc/51461/task/52736/cwd (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52736 ceph-osd ceph rtd unknown /proc/51461/task/52736/root (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52736 ceph-osd ceph txt unknown /proc/51461/task/52736/exe (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52736 ceph-osd ceph NOFD /proc/51461/task/52736/fd (opendir: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52737 osd_srv_a ceph cwd unknown /proc/51461/task/52737/cwd (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52737 osd_srv_a ceph rtd unknown /proc/51461/task/52737/root (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52737 osd_srv_a ceph txt unknown /proc/51461/task/52737/exe (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52737 osd_srv_a ceph NOFD /proc/51461/task/52737/fd (opendir: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52738 tp_osd_tp ceph cwd unknown /proc/51461/task/52738/cwd (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52738 tp_osd_tp ceph rtd unknown /proc/51461/task/52738/root (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52738 tp_osd_tp ceph txt unknown /proc/51461/task/52738/exe (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52738 tp_osd_tp ceph NOFD /proc/51461/task/52738/fd (opendir: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52739 tp_osd_tp ceph cwd unknown /proc/51461/task/52739/cwd (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52739 tp_osd_tp ceph rtd unknown /proc/51461/task/52739/root (readlink: Permission denied) 2024-05-08T05:56:40.996 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52739 tp_osd_tp ceph txt unknown /proc/51461/task/52739/exe (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52739 tp_osd_tp ceph NOFD /proc/51461/task/52739/fd (opendir: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52740 tp_osd_tp ceph cwd unknown /proc/51461/task/52740/cwd (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52740 tp_osd_tp ceph rtd unknown /proc/51461/task/52740/root (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52740 tp_osd_tp ceph txt unknown /proc/51461/task/52740/exe (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52740 tp_osd_tp ceph NOFD /proc/51461/task/52740/fd (opendir: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52741 tp_osd_tp ceph cwd unknown /proc/51461/task/52741/cwd (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52741 tp_osd_tp ceph rtd unknown /proc/51461/task/52741/root (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52741 tp_osd_tp ceph txt unknown /proc/51461/task/52741/exe (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52741 tp_osd_tp ceph NOFD /proc/51461/task/52741/fd (opendir: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52742 tp_osd_tp ceph cwd unknown /proc/51461/task/52742/cwd (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52742 tp_osd_tp ceph rtd unknown /proc/51461/task/52742/root (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52742 tp_osd_tp ceph txt unknown /proc/51461/task/52742/exe (readlink: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52742 tp_osd_tp ceph NOFD /proc/51461/task/52742/fd (opendir: Permission denied) 2024-05-08T05:56:40.997 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52743 tp_osd_tp ceph cwd unknown /proc/51461/task/52743/cwd (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52743 tp_osd_tp ceph rtd unknown /proc/51461/task/52743/root (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52743 tp_osd_tp ceph txt unknown /proc/51461/task/52743/exe (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52743 tp_osd_tp ceph NOFD /proc/51461/task/52743/fd (opendir: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52744 tp_osd_tp ceph cwd unknown /proc/51461/task/52744/cwd (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52744 tp_osd_tp ceph rtd unknown /proc/51461/task/52744/root (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52744 tp_osd_tp ceph txt unknown /proc/51461/task/52744/exe (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52744 tp_osd_tp ceph NOFD /proc/51461/task/52744/fd (opendir: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52745 tp_osd_tp ceph cwd unknown /proc/51461/task/52745/cwd (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52745 tp_osd_tp ceph rtd unknown /proc/51461/task/52745/root (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52745 tp_osd_tp ceph txt unknown /proc/51461/task/52745/exe (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52745 tp_osd_tp ceph NOFD /proc/51461/task/52745/fd (opendir: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52746 tp_osd_tp ceph cwd unknown /proc/51461/task/52746/cwd (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52746 tp_osd_tp ceph rtd unknown /proc/51461/task/52746/root (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52746 tp_osd_tp ceph txt unknown /proc/51461/task/52746/exe (readlink: Permission denied) 2024-05-08T05:56:40.998 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52746 tp_osd_tp ceph NOFD /proc/51461/task/52746/fd (opendir: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52747 tp_osd_tp ceph cwd unknown /proc/51461/task/52747/cwd (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52747 tp_osd_tp ceph rtd unknown /proc/51461/task/52747/root (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52747 tp_osd_tp ceph txt unknown /proc/51461/task/52747/exe (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52747 tp_osd_tp ceph NOFD /proc/51461/task/52747/fd (opendir: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52748 tp_osd_tp ceph cwd unknown /proc/51461/task/52748/cwd (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52748 tp_osd_tp ceph rtd unknown /proc/51461/task/52748/root (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52748 tp_osd_tp ceph txt unknown /proc/51461/task/52748/exe (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52748 tp_osd_tp ceph NOFD /proc/51461/task/52748/fd (opendir: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52749 tp_osd_tp ceph cwd unknown /proc/51461/task/52749/cwd (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52749 tp_osd_tp ceph rtd unknown /proc/51461/task/52749/root (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52749 tp_osd_tp ceph txt unknown /proc/51461/task/52749/exe (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52749 tp_osd_tp ceph NOFD /proc/51461/task/52749/fd (opendir: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52750 tp_osd_tp ceph cwd unknown /proc/51461/task/52750/cwd (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52750 tp_osd_tp ceph rtd unknown /proc/51461/task/52750/root (readlink: Permission denied) 2024-05-08T05:56:40.999 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52750 tp_osd_tp ceph txt unknown /proc/51461/task/52750/exe (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52750 tp_osd_tp ceph NOFD /proc/51461/task/52750/fd (opendir: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52751 tp_osd_tp ceph cwd unknown /proc/51461/task/52751/cwd (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52751 tp_osd_tp ceph rtd unknown /proc/51461/task/52751/root (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52751 tp_osd_tp ceph txt unknown /proc/51461/task/52751/exe (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52751 tp_osd_tp ceph NOFD /proc/51461/task/52751/fd (opendir: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52752 tp_osd_tp ceph cwd unknown /proc/51461/task/52752/cwd (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52752 tp_osd_tp ceph rtd unknown /proc/51461/task/52752/root (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52752 tp_osd_tp ceph txt unknown /proc/51461/task/52752/exe (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52752 tp_osd_tp ceph NOFD /proc/51461/task/52752/fd (opendir: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52753 tp_osd_tp ceph cwd unknown /proc/51461/task/52753/cwd (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52753 tp_osd_tp ceph rtd unknown /proc/51461/task/52753/root (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52753 tp_osd_tp ceph txt unknown /proc/51461/task/52753/exe (readlink: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52753 tp_osd_tp ceph NOFD /proc/51461/task/52753/fd (opendir: Permission denied) 2024-05-08T05:56:41.000 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52754 osd_srv_h ceph cwd unknown /proc/51461/task/52754/cwd (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52754 osd_srv_h ceph rtd unknown /proc/51461/task/52754/root (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52754 osd_srv_h ceph txt unknown /proc/51461/task/52754/exe (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 51461 52754 osd_srv_h ceph NOFD /proc/51461/task/52754/fd (opendir: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 52755 root cwd unknown /proc/52755/cwd (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 52755 root rtd unknown /proc/52755/root (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 52755 root txt unknown /proc/52755/exe (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 52755 root NOFD /proc/52755/fd (opendir: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 52774 root cwd unknown /proc/52774/cwd (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 52774 root rtd unknown /proc/52774/root (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 52774 root txt unknown /proc/52774/exe (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 52774 root NOFD /proc/52774/fd (opendir: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 55547 root cwd unknown /proc/55547/cwd (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 55547 root rtd unknown /proc/55547/root (readlink: Permission denied) 2024-05-08T05:56:41.001 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 55547 root txt unknown /proc/55547/exe (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 55547 root NOFD /proc/55547/fd (opendir: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 58449 root cwd unknown /proc/58449/cwd (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 58449 root rtd unknown /proc/58449/root (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 58449 root txt unknown /proc/58449/exe (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 58449 root NOFD /proc/58449/fd (opendir: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 58451 root cwd unknown /proc/58451/cwd (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 58451 root rtd unknown /proc/58451/root (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 58451 root txt unknown /proc/58451/exe (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 58451 root NOFD /proc/58451/fd (opendir: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 ceph cwd unknown /proc/58453/cwd (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 ceph rtd unknown /proc/58453/root (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 ceph txt unknown /proc/58453/exe (readlink: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 ceph NOFD /proc/58453/fd (opendir: Permission denied) 2024-05-08T05:56:41.002 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58454 log ceph cwd unknown /proc/58453/task/58454/cwd (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58454 log ceph rtd unknown /proc/58453/task/58454/root (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58454 log ceph txt unknown /proc/58453/task/58454/exe (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58454 log ceph NOFD /proc/58453/task/58454/fd (opendir: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58456 msgr-work ceph cwd unknown /proc/58453/task/58456/cwd (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58456 msgr-work ceph rtd unknown /proc/58453/task/58456/root (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58456 msgr-work ceph txt unknown /proc/58453/task/58456/exe (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58456 msgr-work ceph NOFD /proc/58453/task/58456/fd (opendir: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58457 msgr-work ceph cwd unknown /proc/58453/task/58457/cwd (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58457 msgr-work ceph rtd unknown /proc/58453/task/58457/root (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58457 msgr-work ceph txt unknown /proc/58453/task/58457/exe (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58457 msgr-work ceph NOFD /proc/58453/task/58457/fd (opendir: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58458 msgr-work ceph cwd unknown /proc/58453/task/58458/cwd (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58458 msgr-work ceph rtd unknown /proc/58453/task/58458/root (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58458 msgr-work ceph txt unknown /proc/58453/task/58458/exe (readlink: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58458 msgr-work ceph NOFD /proc/58453/task/58458/fd (opendir: Permission denied) 2024-05-08T05:56:41.003 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58462 service ceph cwd unknown /proc/58453/task/58462/cwd (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58462 service ceph rtd unknown /proc/58453/task/58462/root (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58462 service ceph txt unknown /proc/58453/task/58462/exe (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58462 service ceph NOFD /proc/58453/task/58462/fd (opendir: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58463 admin_soc ceph cwd unknown /proc/58453/task/58463/cwd (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58463 admin_soc ceph rtd unknown /proc/58453/task/58463/root (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58463 admin_soc ceph txt unknown /proc/58453/task/58463/exe (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58463 admin_soc ceph NOFD /proc/58453/task/58463/fd (opendir: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58605 signal_ha ceph cwd unknown /proc/58453/task/58605/cwd (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58605 signal_ha ceph rtd unknown /proc/58453/task/58605/root (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58605 signal_ha ceph txt unknown /proc/58453/task/58605/exe (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58605 signal_ha ceph NOFD /proc/58453/task/58605/fd (opendir: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58606 io_contex ceph cwd unknown /proc/58453/task/58606/cwd (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58606 io_contex ceph rtd unknown /proc/58453/task/58606/root (readlink: Permission denied) 2024-05-08T05:56:41.004 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58606 io_contex ceph txt unknown /proc/58453/task/58606/exe (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58606 io_contex ceph NOFD /proc/58453/task/58606/fd (opendir: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58607 io_contex ceph cwd unknown /proc/58453/task/58607/cwd (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58607 io_contex ceph rtd unknown /proc/58453/task/58607/root (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58607 io_contex ceph txt unknown /proc/58453/task/58607/exe (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58607 io_contex ceph NOFD /proc/58453/task/58607/fd (opendir: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58721 OpHistory ceph cwd unknown /proc/58453/task/58721/cwd (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58721 OpHistory ceph rtd unknown /proc/58453/task/58721/root (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58721 OpHistory ceph txt unknown /proc/58453/task/58721/exe (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58721 OpHistory ceph NOFD /proc/58453/task/58721/fd (opendir: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58722 ceph_time ceph cwd unknown /proc/58453/task/58722/cwd (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58722 ceph_time ceph rtd unknown /proc/58453/task/58722/root (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58722 ceph_time ceph txt unknown /proc/58453/task/58722/exe (readlink: Permission denied) 2024-05-08T05:56:41.005 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58722 ceph_time ceph NOFD /proc/58453/task/58722/fd (opendir: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58875 ceph-osd ceph cwd unknown /proc/58453/task/58875/cwd (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58875 ceph-osd ceph rtd unknown /proc/58453/task/58875/root (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58875 ceph-osd ceph txt unknown /proc/58453/task/58875/exe (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58875 ceph-osd ceph NOFD /proc/58453/task/58875/fd (opendir: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58876 safe_time ceph cwd unknown /proc/58453/task/58876/cwd (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58876 safe_time ceph rtd unknown /proc/58453/task/58876/root (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58876 safe_time ceph txt unknown /proc/58453/task/58876/exe (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58876 safe_time ceph NOFD /proc/58453/task/58876/fd (opendir: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58877 safe_time ceph cwd unknown /proc/58453/task/58877/cwd (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58877 safe_time ceph rtd unknown /proc/58453/task/58877/root (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58877 safe_time ceph txt unknown /proc/58453/task/58877/exe (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58877 safe_time ceph NOFD /proc/58453/task/58877/fd (opendir: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58878 safe_time ceph cwd unknown /proc/58453/task/58878/cwd (readlink: Permission denied) 2024-05-08T05:56:41.006 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58878 safe_time ceph rtd unknown /proc/58453/task/58878/root (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58878 safe_time ceph txt unknown /proc/58453/task/58878/exe (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58878 safe_time ceph NOFD /proc/58453/task/58878/fd (opendir: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58879 safe_time ceph cwd unknown /proc/58453/task/58879/cwd (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58879 safe_time ceph rtd unknown /proc/58453/task/58879/root (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58879 safe_time ceph txt unknown /proc/58453/task/58879/exe (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58879 safe_time ceph NOFD /proc/58453/task/58879/fd (opendir: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58881 fn_anonym ceph cwd unknown /proc/58453/task/58881/cwd (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58881 fn_anonym ceph rtd unknown /proc/58453/task/58881/root (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58881 fn_anonym ceph txt unknown /proc/58453/task/58881/exe (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58881 fn_anonym ceph NOFD /proc/58453/task/58881/fd (opendir: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58884 rocksdb:l ceph cwd unknown /proc/58453/task/58884/cwd (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58884 rocksdb:l ceph rtd unknown /proc/58453/task/58884/root (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58884 rocksdb:l ceph txt unknown /proc/58453/task/58884/exe (readlink: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58884 rocksdb:l ceph NOFD /proc/58453/task/58884/fd (opendir: Permission denied) 2024-05-08T05:56:41.007 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58885 rocksdb:l ceph cwd unknown /proc/58453/task/58885/cwd (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58885 rocksdb:l ceph rtd unknown /proc/58453/task/58885/root (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58885 rocksdb:l ceph txt unknown /proc/58453/task/58885/exe (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58885 rocksdb:l ceph NOFD /proc/58453/task/58885/fd (opendir: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58886 rocksdb:l ceph cwd unknown /proc/58453/task/58886/cwd (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58886 rocksdb:l ceph rtd unknown /proc/58453/task/58886/root (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58886 rocksdb:l ceph txt unknown /proc/58453/task/58886/exe (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58886 rocksdb:l ceph NOFD /proc/58453/task/58886/fd (opendir: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58887 rocksdb:h ceph cwd unknown /proc/58453/task/58887/cwd (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58887 rocksdb:h ceph rtd unknown /proc/58453/task/58887/root (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58887 rocksdb:h ceph txt unknown /proc/58453/task/58887/exe (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 58887 rocksdb:h ceph NOFD /proc/58453/task/58887/fd (opendir: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59365 bstore_ai ceph cwd unknown /proc/58453/task/59365/cwd (readlink: Permission denied) 2024-05-08T05:56:41.008 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59365 bstore_ai ceph rtd unknown /proc/58453/task/59365/root (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59365 bstore_ai ceph txt unknown /proc/58453/task/59365/exe (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59365 bstore_ai ceph NOFD /proc/58453/task/59365/fd (opendir: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59550 bstore_ai ceph cwd unknown /proc/58453/task/59550/cwd (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59550 bstore_ai ceph rtd unknown /proc/58453/task/59550/root (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59550 bstore_ai ceph txt unknown /proc/58453/task/59550/exe (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59550 bstore_ai ceph NOFD /proc/58453/task/59550/fd (opendir: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59734 ceph-osd ceph cwd unknown /proc/58453/task/59734/cwd (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59734 ceph-osd ceph rtd unknown /proc/58453/task/59734/root (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59734 ceph-osd ceph txt unknown /proc/58453/task/59734/exe (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59734 ceph-osd ceph NOFD /proc/58453/task/59734/fd (opendir: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59735 cfin ceph cwd unknown /proc/58453/task/59735/cwd (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59735 cfin ceph rtd unknown /proc/58453/task/59735/root (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59735 cfin ceph txt unknown /proc/58453/task/59735/exe (readlink: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59735 cfin ceph NOFD /proc/58453/task/59735/fd (opendir: Permission denied) 2024-05-08T05:56:41.009 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59736 bstore_kv ceph cwd unknown /proc/58453/task/59736/cwd (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59736 bstore_kv ceph rtd unknown /proc/58453/task/59736/root (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59736 bstore_kv ceph txt unknown /proc/58453/task/59736/exe (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59736 bstore_kv ceph NOFD /proc/58453/task/59736/fd (opendir: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59737 bstore_kv ceph cwd unknown /proc/58453/task/59737/cwd (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59737 bstore_kv ceph rtd unknown /proc/58453/task/59737/root (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59737 bstore_kv ceph txt unknown /proc/58453/task/59737/exe (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59737 bstore_kv ceph NOFD /proc/58453/task/59737/fd (opendir: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59738 bstore_me ceph cwd unknown /proc/58453/task/59738/cwd (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59738 bstore_me ceph rtd unknown /proc/58453/task/59738/root (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59738 bstore_me ceph txt unknown /proc/58453/task/59738/exe (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59738 bstore_me ceph NOFD /proc/58453/task/59738/fd (opendir: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59739 ms_dispat ceph cwd unknown /proc/58453/task/59739/cwd (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59739 ms_dispat ceph rtd unknown /proc/58453/task/59739/root (readlink: Permission denied) 2024-05-08T05:56:41.010 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59739 ms_dispat ceph txt unknown /proc/58453/task/59739/exe (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59739 ms_dispat ceph NOFD /proc/58453/task/59739/fd (opendir: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59740 ms_local ceph cwd unknown /proc/58453/task/59740/cwd (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59740 ms_local ceph rtd unknown /proc/58453/task/59740/root (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59740 ms_local ceph txt unknown /proc/58453/task/59740/exe (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59740 ms_local ceph NOFD /proc/58453/task/59740/fd (opendir: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59741 safe_time ceph cwd unknown /proc/58453/task/59741/cwd (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59741 safe_time ceph rtd unknown /proc/58453/task/59741/root (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59741 safe_time ceph txt unknown /proc/58453/task/59741/exe (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59741 safe_time ceph NOFD /proc/58453/task/59741/fd (opendir: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59742 safe_time ceph cwd unknown /proc/58453/task/59742/cwd (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59742 safe_time ceph rtd unknown /proc/58453/task/59742/root (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59742 safe_time ceph txt unknown /proc/58453/task/59742/exe (readlink: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59742 safe_time ceph NOFD /proc/58453/task/59742/fd (opendir: Permission denied) 2024-05-08T05:56:41.011 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59743 ms_dispat ceph cwd unknown /proc/58453/task/59743/cwd (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59743 ms_dispat ceph rtd unknown /proc/58453/task/59743/root (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59743 ms_dispat ceph txt unknown /proc/58453/task/59743/exe (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59743 ms_dispat ceph NOFD /proc/58453/task/59743/fd (opendir: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59744 ms_local ceph cwd unknown /proc/58453/task/59744/cwd (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59744 ms_local ceph rtd unknown /proc/58453/task/59744/root (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59744 ms_local ceph txt unknown /proc/58453/task/59744/exe (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59744 ms_local ceph NOFD /proc/58453/task/59744/fd (opendir: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59745 ms_dispat ceph cwd unknown /proc/58453/task/59745/cwd (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59745 ms_dispat ceph rtd unknown /proc/58453/task/59745/root (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59745 ms_dispat ceph txt unknown /proc/58453/task/59745/exe (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59745 ms_dispat ceph NOFD /proc/58453/task/59745/fd (opendir: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59746 ms_local ceph cwd unknown /proc/58453/task/59746/cwd (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59746 ms_local ceph rtd unknown /proc/58453/task/59746/root (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59746 ms_local ceph txt unknown /proc/58453/task/59746/exe (readlink: Permission denied) 2024-05-08T05:56:41.012 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59746 ms_local ceph NOFD /proc/58453/task/59746/fd (opendir: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59747 ms_dispat ceph cwd unknown /proc/58453/task/59747/cwd (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59747 ms_dispat ceph rtd unknown /proc/58453/task/59747/root (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59747 ms_dispat ceph txt unknown /proc/58453/task/59747/exe (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59747 ms_dispat ceph NOFD /proc/58453/task/59747/fd (opendir: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59748 ms_local ceph cwd unknown /proc/58453/task/59748/cwd (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59748 ms_local ceph rtd unknown /proc/58453/task/59748/root (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59748 ms_local ceph txt unknown /proc/58453/task/59748/exe (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59748 ms_local ceph NOFD /proc/58453/task/59748/fd (opendir: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59749 ms_dispat ceph cwd unknown /proc/58453/task/59749/cwd (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59749 ms_dispat ceph rtd unknown /proc/58453/task/59749/root (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59749 ms_dispat ceph txt unknown /proc/58453/task/59749/exe (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59749 ms_dispat ceph NOFD /proc/58453/task/59749/fd (opendir: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59750 ms_local ceph cwd unknown /proc/58453/task/59750/cwd (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59750 ms_local ceph rtd unknown /proc/58453/task/59750/root (readlink: Permission denied) 2024-05-08T05:56:41.013 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59750 ms_local ceph txt unknown /proc/58453/task/59750/exe (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59750 ms_local ceph NOFD /proc/58453/task/59750/fd (opendir: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59751 ms_dispat ceph cwd unknown /proc/58453/task/59751/cwd (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59751 ms_dispat ceph rtd unknown /proc/58453/task/59751/root (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59751 ms_dispat ceph txt unknown /proc/58453/task/59751/exe (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59751 ms_dispat ceph NOFD /proc/58453/task/59751/fd (opendir: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59752 ms_local ceph cwd unknown /proc/58453/task/59752/cwd (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59752 ms_local ceph rtd unknown /proc/58453/task/59752/root (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59752 ms_local ceph txt unknown /proc/58453/task/59752/exe (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59752 ms_local ceph NOFD /proc/58453/task/59752/fd (opendir: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59753 ms_dispat ceph cwd unknown /proc/58453/task/59753/cwd (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59753 ms_dispat ceph rtd unknown /proc/58453/task/59753/root (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59753 ms_dispat ceph txt unknown /proc/58453/task/59753/exe (readlink: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59753 ms_dispat ceph NOFD /proc/58453/task/59753/fd (opendir: Permission denied) 2024-05-08T05:56:41.014 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59754 ms_local ceph cwd unknown /proc/58453/task/59754/cwd (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59754 ms_local ceph rtd unknown /proc/58453/task/59754/root (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59754 ms_local ceph txt unknown /proc/58453/task/59754/exe (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59754 ms_local ceph NOFD /proc/58453/task/59754/fd (opendir: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59755 fn_anonym ceph cwd unknown /proc/58453/task/59755/cwd (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59755 fn_anonym ceph rtd unknown /proc/58453/task/59755/root (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59755 fn_anonym ceph txt unknown /proc/58453/task/59755/exe (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59755 fn_anonym ceph NOFD /proc/58453/task/59755/fd (opendir: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59756 finisher ceph cwd unknown /proc/58453/task/59756/cwd (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59756 finisher ceph rtd unknown /proc/58453/task/59756/root (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59756 finisher ceph txt unknown /proc/58453/task/59756/exe (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59756 finisher ceph NOFD /proc/58453/task/59756/fd (opendir: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59757 safe_time ceph cwd unknown /proc/58453/task/59757/cwd (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59757 safe_time ceph rtd unknown /proc/58453/task/59757/root (readlink: Permission denied) 2024-05-08T05:56:41.015 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59757 safe_time ceph txt unknown /proc/58453/task/59757/exe (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59757 safe_time ceph NOFD /proc/58453/task/59757/fd (opendir: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59758 safe_time ceph cwd unknown /proc/58453/task/59758/cwd (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59758 safe_time ceph rtd unknown /proc/58453/task/59758/root (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59758 safe_time ceph txt unknown /proc/58453/task/59758/exe (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59758 safe_time ceph NOFD /proc/58453/task/59758/fd (opendir: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59759 ceph-osd ceph cwd unknown /proc/58453/task/59759/cwd (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59759 ceph-osd ceph rtd unknown /proc/58453/task/59759/root (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59759 ceph-osd ceph txt unknown /proc/58453/task/59759/exe (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59759 ceph-osd ceph NOFD /proc/58453/task/59759/fd (opendir: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59760 osd_srv_a ceph cwd unknown /proc/58453/task/59760/cwd (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59760 osd_srv_a ceph rtd unknown /proc/58453/task/59760/root (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59760 osd_srv_a ceph txt unknown /proc/58453/task/59760/exe (readlink: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59760 osd_srv_a ceph NOFD /proc/58453/task/59760/fd (opendir: Permission denied) 2024-05-08T05:56:41.016 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59761 tp_osd_tp ceph cwd unknown /proc/58453/task/59761/cwd (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59761 tp_osd_tp ceph rtd unknown /proc/58453/task/59761/root (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59761 tp_osd_tp ceph txt unknown /proc/58453/task/59761/exe (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59761 tp_osd_tp ceph NOFD /proc/58453/task/59761/fd (opendir: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59762 tp_osd_tp ceph cwd unknown /proc/58453/task/59762/cwd (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59762 tp_osd_tp ceph rtd unknown /proc/58453/task/59762/root (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59762 tp_osd_tp ceph txt unknown /proc/58453/task/59762/exe (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59762 tp_osd_tp ceph NOFD /proc/58453/task/59762/fd (opendir: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59763 tp_osd_tp ceph cwd unknown /proc/58453/task/59763/cwd (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59763 tp_osd_tp ceph rtd unknown /proc/58453/task/59763/root (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59763 tp_osd_tp ceph txt unknown /proc/58453/task/59763/exe (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59763 tp_osd_tp ceph NOFD /proc/58453/task/59763/fd (opendir: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59764 tp_osd_tp ceph cwd unknown /proc/58453/task/59764/cwd (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59764 tp_osd_tp ceph rtd unknown /proc/58453/task/59764/root (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59764 tp_osd_tp ceph txt unknown /proc/58453/task/59764/exe (readlink: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59764 tp_osd_tp ceph NOFD /proc/58453/task/59764/fd (opendir: Permission denied) 2024-05-08T05:56:41.017 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59765 tp_osd_tp ceph cwd unknown /proc/58453/task/59765/cwd (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59765 tp_osd_tp ceph rtd unknown /proc/58453/task/59765/root (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59765 tp_osd_tp ceph txt unknown /proc/58453/task/59765/exe (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59765 tp_osd_tp ceph NOFD /proc/58453/task/59765/fd (opendir: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59766 tp_osd_tp ceph cwd unknown /proc/58453/task/59766/cwd (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59766 tp_osd_tp ceph rtd unknown /proc/58453/task/59766/root (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59766 tp_osd_tp ceph txt unknown /proc/58453/task/59766/exe (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59766 tp_osd_tp ceph NOFD /proc/58453/task/59766/fd (opendir: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59767 tp_osd_tp ceph cwd unknown /proc/58453/task/59767/cwd (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59767 tp_osd_tp ceph rtd unknown /proc/58453/task/59767/root (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59767 tp_osd_tp ceph txt unknown /proc/58453/task/59767/exe (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59767 tp_osd_tp ceph NOFD /proc/58453/task/59767/fd (opendir: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59768 tp_osd_tp ceph cwd unknown /proc/58453/task/59768/cwd (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59768 tp_osd_tp ceph rtd unknown /proc/58453/task/59768/root (readlink: Permission denied) 2024-05-08T05:56:41.018 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59768 tp_osd_tp ceph txt unknown /proc/58453/task/59768/exe (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59768 tp_osd_tp ceph NOFD /proc/58453/task/59768/fd (opendir: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59769 tp_osd_tp ceph cwd unknown /proc/58453/task/59769/cwd (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59769 tp_osd_tp ceph rtd unknown /proc/58453/task/59769/root (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59769 tp_osd_tp ceph txt unknown /proc/58453/task/59769/exe (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59769 tp_osd_tp ceph NOFD /proc/58453/task/59769/fd (opendir: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59770 tp_osd_tp ceph cwd unknown /proc/58453/task/59770/cwd (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59770 tp_osd_tp ceph rtd unknown /proc/58453/task/59770/root (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59770 tp_osd_tp ceph txt unknown /proc/58453/task/59770/exe (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59770 tp_osd_tp ceph NOFD /proc/58453/task/59770/fd (opendir: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59771 tp_osd_tp ceph cwd unknown /proc/58453/task/59771/cwd (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59771 tp_osd_tp ceph rtd unknown /proc/58453/task/59771/root (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59771 tp_osd_tp ceph txt unknown /proc/58453/task/59771/exe (readlink: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59771 tp_osd_tp ceph NOFD /proc/58453/task/59771/fd (opendir: Permission denied) 2024-05-08T05:56:41.019 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59772 tp_osd_tp ceph cwd unknown /proc/58453/task/59772/cwd (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59772 tp_osd_tp ceph rtd unknown /proc/58453/task/59772/root (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59772 tp_osd_tp ceph txt unknown /proc/58453/task/59772/exe (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59772 tp_osd_tp ceph NOFD /proc/58453/task/59772/fd (opendir: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59773 tp_osd_tp ceph cwd unknown /proc/58453/task/59773/cwd (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59773 tp_osd_tp ceph rtd unknown /proc/58453/task/59773/root (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59773 tp_osd_tp ceph txt unknown /proc/58453/task/59773/exe (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59773 tp_osd_tp ceph NOFD /proc/58453/task/59773/fd (opendir: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59774 tp_osd_tp ceph cwd unknown /proc/58453/task/59774/cwd (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59774 tp_osd_tp ceph rtd unknown /proc/58453/task/59774/root (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59774 tp_osd_tp ceph txt unknown /proc/58453/task/59774/exe (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59774 tp_osd_tp ceph NOFD /proc/58453/task/59774/fd (opendir: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59775 tp_osd_tp ceph cwd unknown /proc/58453/task/59775/cwd (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59775 tp_osd_tp ceph rtd unknown /proc/58453/task/59775/root (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59775 tp_osd_tp ceph txt unknown /proc/58453/task/59775/exe (readlink: Permission denied) 2024-05-08T05:56:41.020 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59775 tp_osd_tp ceph NOFD /proc/58453/task/59775/fd (opendir: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59776 tp_osd_tp ceph cwd unknown /proc/58453/task/59776/cwd (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59776 tp_osd_tp ceph rtd unknown /proc/58453/task/59776/root (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59776 tp_osd_tp ceph txt unknown /proc/58453/task/59776/exe (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59776 tp_osd_tp ceph NOFD /proc/58453/task/59776/fd (opendir: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59777 osd_srv_h ceph cwd unknown /proc/58453/task/59777/cwd (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59777 osd_srv_h ceph rtd unknown /proc/58453/task/59777/root (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59777 osd_srv_h ceph txt unknown /proc/58453/task/59777/exe (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 58453 59777 osd_srv_h ceph NOFD /proc/58453/task/59777/fd (opendir: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 59870 root cwd unknown /proc/59870/cwd (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 59870 root rtd unknown /proc/59870/root (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 59870 root txt unknown /proc/59870/exe (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 59870 root NOFD /proc/59870/fd (opendir: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 59887 root cwd unknown /proc/59887/cwd (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 59887 root rtd unknown /proc/59887/root (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 59887 root txt unknown /proc/59887/exe (readlink: Permission denied) 2024-05-08T05:56:41.021 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 59887 root NOFD /proc/59887/fd (opendir: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 62869 root cwd unknown /proc/62869/cwd (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 62869 root rtd unknown /proc/62869/root (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 62869 root txt unknown /proc/62869/exe (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/6 62869 root NOFD /proc/62869/fd (opendir: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 64706 root cwd unknown /proc/64706/cwd (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 64706 root rtd unknown /proc/64706/root (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 64706 root txt unknown /proc/64706/exe (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 64706 root NOFD /proc/64706/fd (opendir: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 65010 root cwd unknown /proc/65010/cwd (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 65010 root rtd unknown /proc/65010/root (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 65010 root txt unknown /proc/65010/exe (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 65010 root NOFD /proc/65010/fd (opendir: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 65457 root cwd unknown /proc/65457/cwd (readlink: Permission denied) 2024-05-08T05:56:41.022 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 65457 root rtd unknown /proc/65457/root (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 65457 root txt unknown /proc/65457/exe (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 65457 root NOFD /proc/65457/fd (opendir: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 65459 root cwd unknown /proc/65459/cwd (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 65459 root rtd unknown /proc/65459/root (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 65459 root txt unknown /proc/65459/exe (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 65459 root NOFD /proc/65459/fd (opendir: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 ceph cwd unknown /proc/65461/cwd (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 ceph rtd unknown /proc/65461/root (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 ceph txt unknown /proc/65461/exe (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 ceph NOFD /proc/65461/fd (opendir: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65462 log ceph cwd unknown /proc/65461/task/65462/cwd (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65462 log ceph rtd unknown /proc/65461/task/65462/root (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65462 log ceph txt unknown /proc/65461/task/65462/exe (readlink: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65462 log ceph NOFD /proc/65461/task/65462/fd (opendir: Permission denied) 2024-05-08T05:56:41.023 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65464 msgr-work ceph cwd unknown /proc/65461/task/65464/cwd (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65464 msgr-work ceph rtd unknown /proc/65461/task/65464/root (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65464 msgr-work ceph txt unknown /proc/65461/task/65464/exe (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65464 msgr-work ceph NOFD /proc/65461/task/65464/fd (opendir: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65465 msgr-work ceph cwd unknown /proc/65461/task/65465/cwd (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65465 msgr-work ceph rtd unknown /proc/65461/task/65465/root (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65465 msgr-work ceph txt unknown /proc/65461/task/65465/exe (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65465 msgr-work ceph NOFD /proc/65461/task/65465/fd (opendir: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65466 msgr-work ceph cwd unknown /proc/65461/task/65466/cwd (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65466 msgr-work ceph rtd unknown /proc/65461/task/65466/root (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65466 msgr-work ceph txt unknown /proc/65461/task/65466/exe (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65466 msgr-work ceph NOFD /proc/65461/task/65466/fd (opendir: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65470 service ceph cwd unknown /proc/65461/task/65470/cwd (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65470 service ceph rtd unknown /proc/65461/task/65470/root (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65470 service ceph txt unknown /proc/65461/task/65470/exe (readlink: Permission denied) 2024-05-08T05:56:41.024 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65470 service ceph NOFD /proc/65461/task/65470/fd (opendir: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65471 admin_soc ceph cwd unknown /proc/65461/task/65471/cwd (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65471 admin_soc ceph rtd unknown /proc/65461/task/65471/root (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65471 admin_soc ceph txt unknown /proc/65461/task/65471/exe (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65471 admin_soc ceph NOFD /proc/65461/task/65471/fd (opendir: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65613 signal_ha ceph cwd unknown /proc/65461/task/65613/cwd (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65613 signal_ha ceph rtd unknown /proc/65461/task/65613/root (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65613 signal_ha ceph txt unknown /proc/65461/task/65613/exe (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65613 signal_ha ceph NOFD /proc/65461/task/65613/fd (opendir: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65614 io_contex ceph cwd unknown /proc/65461/task/65614/cwd (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65614 io_contex ceph rtd unknown /proc/65461/task/65614/root (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65614 io_contex ceph txt unknown /proc/65461/task/65614/exe (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65614 io_contex ceph NOFD /proc/65461/task/65614/fd (opendir: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65615 io_contex ceph cwd unknown /proc/65461/task/65615/cwd (readlink: Permission denied) 2024-05-08T05:56:41.025 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65615 io_contex ceph rtd unknown /proc/65461/task/65615/root (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65615 io_contex ceph txt unknown /proc/65461/task/65615/exe (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65615 io_contex ceph NOFD /proc/65461/task/65615/fd (opendir: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65728 OpHistory ceph cwd unknown /proc/65461/task/65728/cwd (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65728 OpHistory ceph rtd unknown /proc/65461/task/65728/root (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65728 OpHistory ceph txt unknown /proc/65461/task/65728/exe (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65728 OpHistory ceph NOFD /proc/65461/task/65728/fd (opendir: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65729 ceph_time ceph cwd unknown /proc/65461/task/65729/cwd (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65729 ceph_time ceph rtd unknown /proc/65461/task/65729/root (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65729 ceph_time ceph txt unknown /proc/65461/task/65729/exe (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65729 ceph_time ceph NOFD /proc/65461/task/65729/fd (opendir: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65922 ceph-osd ceph cwd unknown /proc/65461/task/65922/cwd (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65922 ceph-osd ceph rtd unknown /proc/65461/task/65922/root (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65922 ceph-osd ceph txt unknown /proc/65461/task/65922/exe (readlink: Permission denied) 2024-05-08T05:56:41.026 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65922 ceph-osd ceph NOFD /proc/65461/task/65922/fd (opendir: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65923 safe_time ceph cwd unknown /proc/65461/task/65923/cwd (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65923 safe_time ceph rtd unknown /proc/65461/task/65923/root (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65923 safe_time ceph txt unknown /proc/65461/task/65923/exe (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65923 safe_time ceph NOFD /proc/65461/task/65923/fd (opendir: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65924 safe_time ceph cwd unknown /proc/65461/task/65924/cwd (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65924 safe_time ceph rtd unknown /proc/65461/task/65924/root (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65924 safe_time ceph txt unknown /proc/65461/task/65924/exe (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65924 safe_time ceph NOFD /proc/65461/task/65924/fd (opendir: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65925 safe_time ceph cwd unknown /proc/65461/task/65925/cwd (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65925 safe_time ceph rtd unknown /proc/65461/task/65925/root (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65925 safe_time ceph txt unknown /proc/65461/task/65925/exe (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65925 safe_time ceph NOFD /proc/65461/task/65925/fd (opendir: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65926 safe_time ceph cwd unknown /proc/65461/task/65926/cwd (readlink: Permission denied) 2024-05-08T05:56:41.027 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65926 safe_time ceph rtd unknown /proc/65461/task/65926/root (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65926 safe_time ceph txt unknown /proc/65461/task/65926/exe (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65926 safe_time ceph NOFD /proc/65461/task/65926/fd (opendir: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65927 fn_anonym ceph cwd unknown /proc/65461/task/65927/cwd (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65927 fn_anonym ceph rtd unknown /proc/65461/task/65927/root (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65927 fn_anonym ceph txt unknown /proc/65461/task/65927/exe (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65927 fn_anonym ceph NOFD /proc/65461/task/65927/fd (opendir: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65935 rocksdb:l ceph cwd unknown /proc/65461/task/65935/cwd (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65935 rocksdb:l ceph rtd unknown /proc/65461/task/65935/root (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65935 rocksdb:l ceph txt unknown /proc/65461/task/65935/exe (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65935 rocksdb:l ceph NOFD /proc/65461/task/65935/fd (opendir: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65936 rocksdb:l ceph cwd unknown /proc/65461/task/65936/cwd (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65936 rocksdb:l ceph rtd unknown /proc/65461/task/65936/root (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65936 rocksdb:l ceph txt unknown /proc/65461/task/65936/exe (readlink: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65936 rocksdb:l ceph NOFD /proc/65461/task/65936/fd (opendir: Permission denied) 2024-05-08T05:56:41.028 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65937 rocksdb:l ceph cwd unknown /proc/65461/task/65937/cwd (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65937 rocksdb:l ceph rtd unknown /proc/65461/task/65937/root (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65937 rocksdb:l ceph txt unknown /proc/65461/task/65937/exe (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65937 rocksdb:l ceph NOFD /proc/65461/task/65937/fd (opendir: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65938 rocksdb:h ceph cwd unknown /proc/65461/task/65938/cwd (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65938 rocksdb:h ceph rtd unknown /proc/65461/task/65938/root (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65938 rocksdb:h ceph txt unknown /proc/65461/task/65938/exe (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 65938 rocksdb:h ceph NOFD /proc/65461/task/65938/fd (opendir: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66397 bstore_ai ceph cwd unknown /proc/65461/task/66397/cwd (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66397 bstore_ai ceph rtd unknown /proc/65461/task/66397/root (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66397 bstore_ai ceph txt unknown /proc/65461/task/66397/exe (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66397 bstore_ai ceph NOFD /proc/65461/task/66397/fd (opendir: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66585 bstore_ai ceph cwd unknown /proc/65461/task/66585/cwd (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66585 bstore_ai ceph rtd unknown /proc/65461/task/66585/root (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66585 bstore_ai ceph txt unknown /proc/65461/task/66585/exe (readlink: Permission denied) 2024-05-08T05:56:41.029 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66585 bstore_ai ceph NOFD /proc/65461/task/66585/fd (opendir: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66768 ceph-osd ceph cwd unknown /proc/65461/task/66768/cwd (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66768 ceph-osd ceph rtd unknown /proc/65461/task/66768/root (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66768 ceph-osd ceph txt unknown /proc/65461/task/66768/exe (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66768 ceph-osd ceph NOFD /proc/65461/task/66768/fd (opendir: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66769 cfin ceph cwd unknown /proc/65461/task/66769/cwd (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66769 cfin ceph rtd unknown /proc/65461/task/66769/root (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66769 cfin ceph txt unknown /proc/65461/task/66769/exe (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66769 cfin ceph NOFD /proc/65461/task/66769/fd (opendir: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66770 bstore_kv ceph cwd unknown /proc/65461/task/66770/cwd (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66770 bstore_kv ceph rtd unknown /proc/65461/task/66770/root (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66770 bstore_kv ceph txt unknown /proc/65461/task/66770/exe (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66770 bstore_kv ceph NOFD /proc/65461/task/66770/fd (opendir: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66771 bstore_kv ceph cwd unknown /proc/65461/task/66771/cwd (readlink: Permission denied) 2024-05-08T05:56:41.030 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66771 bstore_kv ceph rtd unknown /proc/65461/task/66771/root (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66771 bstore_kv ceph txt unknown /proc/65461/task/66771/exe (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66771 bstore_kv ceph NOFD /proc/65461/task/66771/fd (opendir: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66772 bstore_me ceph cwd unknown /proc/65461/task/66772/cwd (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66772 bstore_me ceph rtd unknown /proc/65461/task/66772/root (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66772 bstore_me ceph txt unknown /proc/65461/task/66772/exe (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66772 bstore_me ceph NOFD /proc/65461/task/66772/fd (opendir: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66773 ms_dispat ceph cwd unknown /proc/65461/task/66773/cwd (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66773 ms_dispat ceph rtd unknown /proc/65461/task/66773/root (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66773 ms_dispat ceph txt unknown /proc/65461/task/66773/exe (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66773 ms_dispat ceph NOFD /proc/65461/task/66773/fd (opendir: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66774 ms_local ceph cwd unknown /proc/65461/task/66774/cwd (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66774 ms_local ceph rtd unknown /proc/65461/task/66774/root (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66774 ms_local ceph txt unknown /proc/65461/task/66774/exe (readlink: Permission denied) 2024-05-08T05:56:41.031 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66774 ms_local ceph NOFD /proc/65461/task/66774/fd (opendir: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66775 safe_time ceph cwd unknown /proc/65461/task/66775/cwd (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66775 safe_time ceph rtd unknown /proc/65461/task/66775/root (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66775 safe_time ceph txt unknown /proc/65461/task/66775/exe (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66775 safe_time ceph NOFD /proc/65461/task/66775/fd (opendir: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66776 safe_time ceph cwd unknown /proc/65461/task/66776/cwd (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66776 safe_time ceph rtd unknown /proc/65461/task/66776/root (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66776 safe_time ceph txt unknown /proc/65461/task/66776/exe (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66776 safe_time ceph NOFD /proc/65461/task/66776/fd (opendir: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66777 ms_dispat ceph cwd unknown /proc/65461/task/66777/cwd (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66777 ms_dispat ceph rtd unknown /proc/65461/task/66777/root (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66777 ms_dispat ceph txt unknown /proc/65461/task/66777/exe (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66777 ms_dispat ceph NOFD /proc/65461/task/66777/fd (opendir: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66778 ms_local ceph cwd unknown /proc/65461/task/66778/cwd (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66778 ms_local ceph rtd unknown /proc/65461/task/66778/root (readlink: Permission denied) 2024-05-08T05:56:41.032 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66778 ms_local ceph txt unknown /proc/65461/task/66778/exe (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66778 ms_local ceph NOFD /proc/65461/task/66778/fd (opendir: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66779 ms_dispat ceph cwd unknown /proc/65461/task/66779/cwd (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66779 ms_dispat ceph rtd unknown /proc/65461/task/66779/root (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66779 ms_dispat ceph txt unknown /proc/65461/task/66779/exe (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66779 ms_dispat ceph NOFD /proc/65461/task/66779/fd (opendir: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66780 ms_local ceph cwd unknown /proc/65461/task/66780/cwd (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66780 ms_local ceph rtd unknown /proc/65461/task/66780/root (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66780 ms_local ceph txt unknown /proc/65461/task/66780/exe (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66780 ms_local ceph NOFD /proc/65461/task/66780/fd (opendir: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66781 ms_dispat ceph cwd unknown /proc/65461/task/66781/cwd (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66781 ms_dispat ceph rtd unknown /proc/65461/task/66781/root (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66781 ms_dispat ceph txt unknown /proc/65461/task/66781/exe (readlink: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66781 ms_dispat ceph NOFD /proc/65461/task/66781/fd (opendir: Permission denied) 2024-05-08T05:56:41.033 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66782 ms_local ceph cwd unknown /proc/65461/task/66782/cwd (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66782 ms_local ceph rtd unknown /proc/65461/task/66782/root (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66782 ms_local ceph txt unknown /proc/65461/task/66782/exe (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66782 ms_local ceph NOFD /proc/65461/task/66782/fd (opendir: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66783 ms_dispat ceph cwd unknown /proc/65461/task/66783/cwd (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66783 ms_dispat ceph rtd unknown /proc/65461/task/66783/root (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66783 ms_dispat ceph txt unknown /proc/65461/task/66783/exe (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66783 ms_dispat ceph NOFD /proc/65461/task/66783/fd (opendir: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66784 ms_local ceph cwd unknown /proc/65461/task/66784/cwd (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66784 ms_local ceph rtd unknown /proc/65461/task/66784/root (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66784 ms_local ceph txt unknown /proc/65461/task/66784/exe (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66784 ms_local ceph NOFD /proc/65461/task/66784/fd (opendir: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66785 ms_dispat ceph cwd unknown /proc/65461/task/66785/cwd (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66785 ms_dispat ceph rtd unknown /proc/65461/task/66785/root (readlink: Permission denied) 2024-05-08T05:56:41.034 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66785 ms_dispat ceph txt unknown /proc/65461/task/66785/exe (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66785 ms_dispat ceph NOFD /proc/65461/task/66785/fd (opendir: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66786 ms_local ceph cwd unknown /proc/65461/task/66786/cwd (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66786 ms_local ceph rtd unknown /proc/65461/task/66786/root (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66786 ms_local ceph txt unknown /proc/65461/task/66786/exe (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66786 ms_local ceph NOFD /proc/65461/task/66786/fd (opendir: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66787 ms_dispat ceph cwd unknown /proc/65461/task/66787/cwd (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66787 ms_dispat ceph rtd unknown /proc/65461/task/66787/root (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66787 ms_dispat ceph txt unknown /proc/65461/task/66787/exe (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66787 ms_dispat ceph NOFD /proc/65461/task/66787/fd (opendir: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66788 ms_local ceph cwd unknown /proc/65461/task/66788/cwd (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66788 ms_local ceph rtd unknown /proc/65461/task/66788/root (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66788 ms_local ceph txt unknown /proc/65461/task/66788/exe (readlink: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66788 ms_local ceph NOFD /proc/65461/task/66788/fd (opendir: Permission denied) 2024-05-08T05:56:41.035 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66789 fn_anonym ceph cwd unknown /proc/65461/task/66789/cwd (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66789 fn_anonym ceph rtd unknown /proc/65461/task/66789/root (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66789 fn_anonym ceph txt unknown /proc/65461/task/66789/exe (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66789 fn_anonym ceph NOFD /proc/65461/task/66789/fd (opendir: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66790 finisher ceph cwd unknown /proc/65461/task/66790/cwd (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66790 finisher ceph rtd unknown /proc/65461/task/66790/root (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66790 finisher ceph txt unknown /proc/65461/task/66790/exe (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66790 finisher ceph NOFD /proc/65461/task/66790/fd (opendir: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66791 safe_time ceph cwd unknown /proc/65461/task/66791/cwd (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66791 safe_time ceph rtd unknown /proc/65461/task/66791/root (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66791 safe_time ceph txt unknown /proc/65461/task/66791/exe (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66791 safe_time ceph NOFD /proc/65461/task/66791/fd (opendir: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66792 safe_time ceph cwd unknown /proc/65461/task/66792/cwd (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66792 safe_time ceph rtd unknown /proc/65461/task/66792/root (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66792 safe_time ceph txt unknown /proc/65461/task/66792/exe (readlink: Permission denied) 2024-05-08T05:56:41.036 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66792 safe_time ceph NOFD /proc/65461/task/66792/fd (opendir: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66793 ceph-osd ceph cwd unknown /proc/65461/task/66793/cwd (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66793 ceph-osd ceph rtd unknown /proc/65461/task/66793/root (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66793 ceph-osd ceph txt unknown /proc/65461/task/66793/exe (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66793 ceph-osd ceph NOFD /proc/65461/task/66793/fd (opendir: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66794 osd_srv_a ceph cwd unknown /proc/65461/task/66794/cwd (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66794 osd_srv_a ceph rtd unknown /proc/65461/task/66794/root (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66794 osd_srv_a ceph txt unknown /proc/65461/task/66794/exe (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66794 osd_srv_a ceph NOFD /proc/65461/task/66794/fd (opendir: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66795 tp_osd_tp ceph cwd unknown /proc/65461/task/66795/cwd (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66795 tp_osd_tp ceph rtd unknown /proc/65461/task/66795/root (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66795 tp_osd_tp ceph txt unknown /proc/65461/task/66795/exe (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66795 tp_osd_tp ceph NOFD /proc/65461/task/66795/fd (opendir: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66796 tp_osd_tp ceph cwd unknown /proc/65461/task/66796/cwd (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66796 tp_osd_tp ceph rtd unknown /proc/65461/task/66796/root (readlink: Permission denied) 2024-05-08T05:56:41.037 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66796 tp_osd_tp ceph txt unknown /proc/65461/task/66796/exe (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66796 tp_osd_tp ceph NOFD /proc/65461/task/66796/fd (opendir: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66797 tp_osd_tp ceph cwd unknown /proc/65461/task/66797/cwd (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66797 tp_osd_tp ceph rtd unknown /proc/65461/task/66797/root (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66797 tp_osd_tp ceph txt unknown /proc/65461/task/66797/exe (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66797 tp_osd_tp ceph NOFD /proc/65461/task/66797/fd (opendir: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66798 tp_osd_tp ceph cwd unknown /proc/65461/task/66798/cwd (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66798 tp_osd_tp ceph rtd unknown /proc/65461/task/66798/root (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66798 tp_osd_tp ceph txt unknown /proc/65461/task/66798/exe (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66798 tp_osd_tp ceph NOFD /proc/65461/task/66798/fd (opendir: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66799 tp_osd_tp ceph cwd unknown /proc/65461/task/66799/cwd (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66799 tp_osd_tp ceph rtd unknown /proc/65461/task/66799/root (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66799 tp_osd_tp ceph txt unknown /proc/65461/task/66799/exe (readlink: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66799 tp_osd_tp ceph NOFD /proc/65461/task/66799/fd (opendir: Permission denied) 2024-05-08T05:56:41.038 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66800 tp_osd_tp ceph cwd unknown /proc/65461/task/66800/cwd (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66800 tp_osd_tp ceph rtd unknown /proc/65461/task/66800/root (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66800 tp_osd_tp ceph txt unknown /proc/65461/task/66800/exe (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66800 tp_osd_tp ceph NOFD /proc/65461/task/66800/fd (opendir: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66801 tp_osd_tp ceph cwd unknown /proc/65461/task/66801/cwd (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66801 tp_osd_tp ceph rtd unknown /proc/65461/task/66801/root (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66801 tp_osd_tp ceph txt unknown /proc/65461/task/66801/exe (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66801 tp_osd_tp ceph NOFD /proc/65461/task/66801/fd (opendir: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66802 tp_osd_tp ceph cwd unknown /proc/65461/task/66802/cwd (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66802 tp_osd_tp ceph rtd unknown /proc/65461/task/66802/root (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66802 tp_osd_tp ceph txt unknown /proc/65461/task/66802/exe (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66802 tp_osd_tp ceph NOFD /proc/65461/task/66802/fd (opendir: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66803 tp_osd_tp ceph cwd unknown /proc/65461/task/66803/cwd (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66803 tp_osd_tp ceph rtd unknown /proc/65461/task/66803/root (readlink: Permission denied) 2024-05-08T05:56:41.039 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66803 tp_osd_tp ceph txt unknown /proc/65461/task/66803/exe (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66803 tp_osd_tp ceph NOFD /proc/65461/task/66803/fd (opendir: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66804 tp_osd_tp ceph cwd unknown /proc/65461/task/66804/cwd (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66804 tp_osd_tp ceph rtd unknown /proc/65461/task/66804/root (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66804 tp_osd_tp ceph txt unknown /proc/65461/task/66804/exe (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66804 tp_osd_tp ceph NOFD /proc/65461/task/66804/fd (opendir: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66805 tp_osd_tp ceph cwd unknown /proc/65461/task/66805/cwd (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66805 tp_osd_tp ceph rtd unknown /proc/65461/task/66805/root (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66805 tp_osd_tp ceph txt unknown /proc/65461/task/66805/exe (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66805 tp_osd_tp ceph NOFD /proc/65461/task/66805/fd (opendir: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66806 tp_osd_tp ceph cwd unknown /proc/65461/task/66806/cwd (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66806 tp_osd_tp ceph rtd unknown /proc/65461/task/66806/root (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66806 tp_osd_tp ceph txt unknown /proc/65461/task/66806/exe (readlink: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66806 tp_osd_tp ceph NOFD /proc/65461/task/66806/fd (opendir: Permission denied) 2024-05-08T05:56:41.040 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66807 tp_osd_tp ceph cwd unknown /proc/65461/task/66807/cwd (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66807 tp_osd_tp ceph rtd unknown /proc/65461/task/66807/root (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66807 tp_osd_tp ceph txt unknown /proc/65461/task/66807/exe (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66807 tp_osd_tp ceph NOFD /proc/65461/task/66807/fd (opendir: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66808 tp_osd_tp ceph cwd unknown /proc/65461/task/66808/cwd (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66808 tp_osd_tp ceph rtd unknown /proc/65461/task/66808/root (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66808 tp_osd_tp ceph txt unknown /proc/65461/task/66808/exe (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66808 tp_osd_tp ceph NOFD /proc/65461/task/66808/fd (opendir: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66809 tp_osd_tp ceph cwd unknown /proc/65461/task/66809/cwd (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66809 tp_osd_tp ceph rtd unknown /proc/65461/task/66809/root (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66809 tp_osd_tp ceph txt unknown /proc/65461/task/66809/exe (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66809 tp_osd_tp ceph NOFD /proc/65461/task/66809/fd (opendir: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66810 tp_osd_tp ceph cwd unknown /proc/65461/task/66810/cwd (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66810 tp_osd_tp ceph rtd unknown /proc/65461/task/66810/root (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66810 tp_osd_tp ceph txt unknown /proc/65461/task/66810/exe (readlink: Permission denied) 2024-05-08T05:56:41.041 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66810 tp_osd_tp ceph NOFD /proc/65461/task/66810/fd (opendir: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66811 osd_srv_h ceph cwd unknown /proc/65461/task/66811/cwd (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66811 osd_srv_h ceph rtd unknown /proc/65461/task/66811/root (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66811 osd_srv_h ceph txt unknown /proc/65461/task/66811/exe (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-osd 65461 66811 osd_srv_h ceph NOFD /proc/65461/task/66811/fd (opendir: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 66905 root cwd unknown /proc/66905/cwd (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 66905 root rtd unknown /proc/66905/root (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 66905 root txt unknown /proc/66905/exe (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 66905 root NOFD /proc/66905/fd (opendir: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 66922 root cwd unknown /proc/66922/cwd (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 66922 root rtd unknown /proc/66922/root (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 66922 root txt unknown /proc/66922/exe (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 66922 root NOFD /proc/66922/fd (opendir: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69597 root cwd unknown /proc/69597/cwd (readlink: Permission denied) 2024-05-08T05:56:41.042 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69597 root rtd unknown /proc/69597/root (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69597 root txt unknown /proc/69597/exe (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69597 root NOFD /proc/69597/fd (opendir: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69598 root cwd unknown /proc/69598/cwd (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69598 root rtd unknown /proc/69598/root (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69598 root txt unknown /proc/69598/exe (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 69598 root NOFD /proc/69598/fd (opendir: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70223 root cwd unknown /proc/70223/cwd (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70223 root rtd unknown /proc/70223/root (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70223 root txt unknown /proc/70223/exe (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70223 root NOFD /proc/70223/fd (opendir: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70240 root cwd unknown /proc/70240/cwd (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70240 root rtd unknown /proc/70240/root (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70240 root txt unknown /proc/70240/exe (readlink: Permission denied) 2024-05-08T05:56:41.043 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70240 root NOFD /proc/70240/fd (opendir: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70241 root cwd unknown /proc/70241/cwd (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70241 root rtd unknown /proc/70241/root (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70241 root txt unknown /proc/70241/exe (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70241 root NOFD /proc/70241/fd (opendir: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70258 root cwd unknown /proc/70258/cwd (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70258 root rtd unknown /proc/70258/root (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70258 root txt unknown /proc/70258/exe (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70258 root NOFD /proc/70258/fd (opendir: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70259 root cwd unknown /proc/70259/cwd (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70259 root rtd unknown /proc/70259/root (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70259 root txt unknown /proc/70259/exe (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70259 root NOFD /proc/70259/fd (opendir: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70276 root cwd unknown /proc/70276/cwd (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70276 root rtd unknown /proc/70276/root (readlink: Permission denied) 2024-05-08T05:56:41.044 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70276 root txt unknown /proc/70276/exe (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70276 root NOFD /proc/70276/fd (opendir: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70277 root cwd unknown /proc/70277/cwd (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70277 root rtd unknown /proc/70277/root (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70277 root txt unknown /proc/70277/exe (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 70277 root NOFD /proc/70277/fd (opendir: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70294 root cwd unknown /proc/70294/cwd (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70294 root rtd unknown /proc/70294/root (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70294 root txt unknown /proc/70294/exe (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:journalct 70294 root NOFD /proc/70294/fd (opendir: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 70887 root cwd unknown /proc/70887/cwd (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 70887 root rtd unknown /proc/70887/root (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 70887 root txt unknown /proc/70887/exe (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 70887 root NOFD /proc/70887/fd (opendir: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 70889 root cwd unknown /proc/70889/cwd (readlink: Permission denied) 2024-05-08T05:56:41.045 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 70889 root rtd unknown /proc/70889/root (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 70889 root txt unknown /proc/70889/exe (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 70889 root NOFD /proc/70889/fd (opendir: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 ceph cwd unknown /proc/70891/cwd (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 ceph rtd unknown /proc/70891/root (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 ceph txt unknown /proc/70891/exe (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 ceph NOFD /proc/70891/fd (opendir: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70892 log ceph cwd unknown /proc/70891/task/70892/cwd (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70892 log ceph rtd unknown /proc/70891/task/70892/root (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70892 log ceph txt unknown /proc/70891/task/70892/exe (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70892 log ceph NOFD /proc/70891/task/70892/fd (opendir: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70894 msgr-work ceph cwd unknown /proc/70891/task/70894/cwd (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70894 msgr-work ceph rtd unknown /proc/70891/task/70894/root (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70894 msgr-work ceph txt unknown /proc/70891/task/70894/exe (readlink: Permission denied) 2024-05-08T05:56:41.046 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70894 msgr-work ceph NOFD /proc/70891/task/70894/fd (opendir: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70895 msgr-work ceph cwd unknown /proc/70891/task/70895/cwd (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70895 msgr-work ceph rtd unknown /proc/70891/task/70895/root (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70895 msgr-work ceph txt unknown /proc/70891/task/70895/exe (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70895 msgr-work ceph NOFD /proc/70891/task/70895/fd (opendir: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70896 msgr-work ceph cwd unknown /proc/70891/task/70896/cwd (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70896 msgr-work ceph rtd unknown /proc/70891/task/70896/root (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70896 msgr-work ceph txt unknown /proc/70891/task/70896/exe (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70896 msgr-work ceph NOFD /proc/70891/task/70896/fd (opendir: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70900 service ceph cwd unknown /proc/70891/task/70900/cwd (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70900 service ceph rtd unknown /proc/70891/task/70900/root (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70900 service ceph txt unknown /proc/70891/task/70900/exe (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70900 service ceph NOFD /proc/70891/task/70900/fd (opendir: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70901 admin_soc ceph cwd unknown /proc/70891/task/70901/cwd (readlink: Permission denied) 2024-05-08T05:56:41.047 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70901 admin_soc ceph rtd unknown /proc/70891/task/70901/root (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70901 admin_soc ceph txt unknown /proc/70891/task/70901/exe (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70901 admin_soc ceph NOFD /proc/70891/task/70901/fd (opendir: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70902 signal_ha ceph cwd unknown /proc/70891/task/70902/cwd (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70902 signal_ha ceph rtd unknown /proc/70891/task/70902/root (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70902 signal_ha ceph txt unknown /proc/70891/task/70902/exe (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70902 signal_ha ceph NOFD /proc/70891/task/70902/fd (opendir: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70903 io_contex ceph cwd unknown /proc/70891/task/70903/cwd (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70903 io_contex ceph rtd unknown /proc/70891/task/70903/root (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70903 io_contex ceph txt unknown /proc/70891/task/70903/exe (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70903 io_contex ceph NOFD /proc/70891/task/70903/fd (opendir: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70904 io_contex ceph cwd unknown /proc/70891/task/70904/cwd (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70904 io_contex ceph rtd unknown /proc/70891/task/70904/root (readlink: Permission denied) 2024-05-08T05:56:41.048 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70904 io_contex ceph txt unknown /proc/70891/task/70904/exe (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70904 io_contex ceph NOFD /proc/70891/task/70904/fd (opendir: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70905 ms_dispat ceph cwd unknown /proc/70891/task/70905/cwd (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70905 ms_dispat ceph rtd unknown /proc/70891/task/70905/root (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70905 ms_dispat ceph txt unknown /proc/70891/task/70905/exe (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70905 ms_dispat ceph NOFD /proc/70891/task/70905/fd (opendir: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70906 ms_local ceph cwd unknown /proc/70891/task/70906/cwd (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70906 ms_local ceph rtd unknown /proc/70891/task/70906/root (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70906 ms_local ceph txt unknown /proc/70891/task/70906/exe (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70906 ms_local ceph NOFD /proc/70891/task/70906/fd (opendir: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70907 safe_time ceph cwd unknown /proc/70891/task/70907/cwd (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70907 safe_time ceph rtd unknown /proc/70891/task/70907/root (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70907 safe_time ceph txt unknown /proc/70891/task/70907/exe (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70907 safe_time ceph NOFD /proc/70891/task/70907/fd (opendir: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70908 safe_time ceph cwd unknown /proc/70891/task/70908/cwd (readlink: Permission denied) 2024-05-08T05:56:41.049 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70908 safe_time ceph rtd unknown /proc/70891/task/70908/root (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70908 safe_time ceph txt unknown /proc/70891/task/70908/exe (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70908 safe_time ceph NOFD /proc/70891/task/70908/fd (opendir: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70909 ceph-mds ceph cwd unknown /proc/70891/task/70909/cwd (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70909 ceph-mds ceph rtd unknown /proc/70891/task/70909/root (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70909 ceph-mds ceph txt unknown /proc/70891/task/70909/exe (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 70909 ceph-mds ceph NOFD /proc/70891/task/70909/fd (opendir: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 71096 safe_time ceph cwd unknown /proc/70891/task/71096/cwd (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 71096 safe_time ceph rtd unknown /proc/70891/task/71096/root (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 71096 safe_time ceph txt unknown /proc/70891/task/71096/exe (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 70891 71096 safe_time ceph NOFD /proc/70891/task/71096/fd (opendir: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 71348 root cwd unknown /proc/71348/cwd (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 71348 root rtd unknown /proc/71348/root (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 71348 root txt unknown /proc/71348/exe (readlink: Permission denied) 2024-05-08T05:56:41.050 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 71348 root NOFD /proc/71348/fd (opendir: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 71350 root cwd unknown /proc/71350/cwd (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 71350 root rtd unknown /proc/71350/root (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 71350 root txt unknown /proc/71350/exe (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 71350 root NOFD /proc/71350/fd (opendir: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 ceph cwd unknown /proc/71353/cwd (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 ceph rtd unknown /proc/71353/root (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 ceph txt unknown /proc/71353/exe (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 ceph NOFD /proc/71353/fd (opendir: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71354 log ceph cwd unknown /proc/71353/task/71354/cwd (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71354 log ceph rtd unknown /proc/71353/task/71354/root (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71354 log ceph txt unknown /proc/71353/task/71354/exe (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71354 log ceph NOFD /proc/71353/task/71354/fd (opendir: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71356 msgr-work ceph cwd unknown /proc/71353/task/71356/cwd (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71356 msgr-work ceph rtd unknown /proc/71353/task/71356/root (readlink: Permission denied) 2024-05-08T05:56:41.051 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71356 msgr-work ceph txt unknown /proc/71353/task/71356/exe (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71356 msgr-work ceph NOFD /proc/71353/task/71356/fd (opendir: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71357 msgr-work ceph cwd unknown /proc/71353/task/71357/cwd (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71357 msgr-work ceph rtd unknown /proc/71353/task/71357/root (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71357 msgr-work ceph txt unknown /proc/71353/task/71357/exe (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71357 msgr-work ceph NOFD /proc/71353/task/71357/fd (opendir: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71358 msgr-work ceph cwd unknown /proc/71353/task/71358/cwd (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71358 msgr-work ceph rtd unknown /proc/71353/task/71358/root (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71358 msgr-work ceph txt unknown /proc/71353/task/71358/exe (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71358 msgr-work ceph NOFD /proc/71353/task/71358/fd (opendir: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71362 service ceph cwd unknown /proc/71353/task/71362/cwd (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71362 service ceph rtd unknown /proc/71353/task/71362/root (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71362 service ceph txt unknown /proc/71353/task/71362/exe (readlink: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71362 service ceph NOFD /proc/71353/task/71362/fd (opendir: Permission denied) 2024-05-08T05:56:41.052 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71363 admin_soc ceph cwd unknown /proc/71353/task/71363/cwd (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71363 admin_soc ceph rtd unknown /proc/71353/task/71363/root (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71363 admin_soc ceph txt unknown /proc/71353/task/71363/exe (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71363 admin_soc ceph NOFD /proc/71353/task/71363/fd (opendir: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71364 signal_ha ceph cwd unknown /proc/71353/task/71364/cwd (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71364 signal_ha ceph rtd unknown /proc/71353/task/71364/root (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71364 signal_ha ceph txt unknown /proc/71353/task/71364/exe (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71364 signal_ha ceph NOFD /proc/71353/task/71364/fd (opendir: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71365 io_contex ceph cwd unknown /proc/71353/task/71365/cwd (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71365 io_contex ceph rtd unknown /proc/71353/task/71365/root (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71365 io_contex ceph txt unknown /proc/71353/task/71365/exe (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71365 io_contex ceph NOFD /proc/71353/task/71365/fd (opendir: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71366 io_contex ceph cwd unknown /proc/71353/task/71366/cwd (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71366 io_contex ceph rtd unknown /proc/71353/task/71366/root (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71366 io_contex ceph txt unknown /proc/71353/task/71366/exe (readlink: Permission denied) 2024-05-08T05:56:41.053 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71366 io_contex ceph NOFD /proc/71353/task/71366/fd (opendir: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71367 ms_dispat ceph cwd unknown /proc/71353/task/71367/cwd (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71367 ms_dispat ceph rtd unknown /proc/71353/task/71367/root (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71367 ms_dispat ceph txt unknown /proc/71353/task/71367/exe (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71367 ms_dispat ceph NOFD /proc/71353/task/71367/fd (opendir: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71368 ms_local ceph cwd unknown /proc/71353/task/71368/cwd (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71368 ms_local ceph rtd unknown /proc/71353/task/71368/root (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71368 ms_local ceph txt unknown /proc/71353/task/71368/exe (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71368 ms_local ceph NOFD /proc/71353/task/71368/fd (opendir: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71369 safe_time ceph cwd unknown /proc/71353/task/71369/cwd (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71369 safe_time ceph rtd unknown /proc/71353/task/71369/root (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71369 safe_time ceph txt unknown /proc/71353/task/71369/exe (readlink: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71369 safe_time ceph NOFD /proc/71353/task/71369/fd (opendir: Permission denied) 2024-05-08T05:56:41.054 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71370 safe_time ceph cwd unknown /proc/71353/task/71370/cwd (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71370 safe_time ceph rtd unknown /proc/71353/task/71370/root (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71370 safe_time ceph txt unknown /proc/71353/task/71370/exe (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71370 safe_time ceph NOFD /proc/71353/task/71370/fd (opendir: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71371 ceph-mds ceph cwd unknown /proc/71353/task/71371/cwd (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71371 ceph-mds ceph rtd unknown /proc/71353/task/71371/root (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71371 ceph-mds ceph txt unknown /proc/71353/task/71371/exe (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71371 ceph-mds ceph NOFD /proc/71353/task/71371/fd (opendir: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71376 safe_time ceph cwd unknown /proc/71353/task/71376/cwd (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71376 safe_time ceph rtd unknown /proc/71353/task/71376/root (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71376 safe_time ceph txt unknown /proc/71353/task/71376/exe (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 71353 71376 safe_time ceph NOFD /proc/71353/task/71376/fd (opendir: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 71533 root cwd unknown /proc/71533/cwd (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 71533 root rtd unknown /proc/71533/root (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 71533 root txt unknown /proc/71533/exe (readlink: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 71533 root NOFD /proc/71533/fd (opendir: Permission denied) 2024-05-08T05:56:41.055 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72160 root cwd unknown /proc/72160/cwd (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72160 root rtd unknown /proc/72160/root (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72160 root txt unknown /proc/72160/exe (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72160 root NOFD /proc/72160/fd (opendir: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72162 root cwd unknown /proc/72162/cwd (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72162 root rtd unknown /proc/72162/root (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72162 root txt unknown /proc/72162/exe (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72162 root NOFD /proc/72162/fd (opendir: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 ceph cwd unknown /proc/72177/cwd (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 ceph rtd unknown /proc/72177/root (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 ceph txt unknown /proc/72177/exe (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 ceph NOFD /proc/72177/fd (opendir: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72178 log ceph cwd unknown /proc/72177/task/72178/cwd (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72178 log ceph rtd unknown /proc/72177/task/72178/root (readlink: Permission denied) 2024-05-08T05:56:41.056 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72178 log ceph txt unknown /proc/72177/task/72178/exe (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72178 log ceph NOFD /proc/72177/task/72178/fd (opendir: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72180 msgr-work ceph cwd unknown /proc/72177/task/72180/cwd (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72180 msgr-work ceph rtd unknown /proc/72177/task/72180/root (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72180 msgr-work ceph txt unknown /proc/72177/task/72180/exe (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72180 msgr-work ceph NOFD /proc/72177/task/72180/fd (opendir: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72181 msgr-work ceph cwd unknown /proc/72177/task/72181/cwd (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72181 msgr-work ceph rtd unknown /proc/72177/task/72181/root (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72181 msgr-work ceph txt unknown /proc/72177/task/72181/exe (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72181 msgr-work ceph NOFD /proc/72177/task/72181/fd (opendir: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72182 msgr-work ceph cwd unknown /proc/72177/task/72182/cwd (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72182 msgr-work ceph rtd unknown /proc/72177/task/72182/root (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72182 msgr-work ceph txt unknown /proc/72177/task/72182/exe (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72182 msgr-work ceph NOFD /proc/72177/task/72182/fd (opendir: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72186 service ceph cwd unknown /proc/72177/task/72186/cwd (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72186 service ceph rtd unknown /proc/72177/task/72186/root (readlink: Permission denied) 2024-05-08T05:56:41.057 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72186 service ceph txt unknown /proc/72177/task/72186/exe (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72186 service ceph NOFD /proc/72177/task/72186/fd (opendir: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72187 admin_soc ceph cwd unknown /proc/72177/task/72187/cwd (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72187 admin_soc ceph rtd unknown /proc/72177/task/72187/root (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72187 admin_soc ceph txt unknown /proc/72177/task/72187/exe (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72187 admin_soc ceph NOFD /proc/72177/task/72187/fd (opendir: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72188 signal_ha ceph cwd unknown /proc/72177/task/72188/cwd (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72188 signal_ha ceph rtd unknown /proc/72177/task/72188/root (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72188 signal_ha ceph txt unknown /proc/72177/task/72188/exe (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72188 signal_ha ceph NOFD /proc/72177/task/72188/fd (opendir: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72189 io_contex ceph cwd unknown /proc/72177/task/72189/cwd (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72189 io_contex ceph rtd unknown /proc/72177/task/72189/root (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72189 io_contex ceph txt unknown /proc/72177/task/72189/exe (readlink: Permission denied) 2024-05-08T05:56:41.058 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72189 io_contex ceph NOFD /proc/72177/task/72189/fd (opendir: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72190 io_contex ceph cwd unknown /proc/72177/task/72190/cwd (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72190 io_contex ceph rtd unknown /proc/72177/task/72190/root (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72190 io_contex ceph txt unknown /proc/72177/task/72190/exe (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72190 io_contex ceph NOFD /proc/72177/task/72190/fd (opendir: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72191 ms_dispat ceph cwd unknown /proc/72177/task/72191/cwd (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72191 ms_dispat ceph rtd unknown /proc/72177/task/72191/root (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72191 ms_dispat ceph txt unknown /proc/72177/task/72191/exe (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72191 ms_dispat ceph NOFD /proc/72177/task/72191/fd (opendir: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72192 ms_local ceph cwd unknown /proc/72177/task/72192/cwd (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72192 ms_local ceph rtd unknown /proc/72177/task/72192/root (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72192 ms_local ceph txt unknown /proc/72177/task/72192/exe (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72192 ms_local ceph NOFD /proc/72177/task/72192/fd (opendir: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72193 safe_time ceph cwd unknown /proc/72177/task/72193/cwd (readlink: Permission denied) 2024-05-08T05:56:41.059 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72193 safe_time ceph rtd unknown /proc/72177/task/72193/root (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72193 safe_time ceph txt unknown /proc/72177/task/72193/exe (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72193 safe_time ceph NOFD /proc/72177/task/72193/fd (opendir: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72194 safe_time ceph cwd unknown /proc/72177/task/72194/cwd (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72194 safe_time ceph rtd unknown /proc/72177/task/72194/root (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72194 safe_time ceph txt unknown /proc/72177/task/72194/exe (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72194 safe_time ceph NOFD /proc/72177/task/72194/fd (opendir: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72195 ceph-mds ceph cwd unknown /proc/72177/task/72195/cwd (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72195 ceph-mds ceph rtd unknown /proc/72177/task/72195/root (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72195 ceph-mds ceph txt unknown /proc/72177/task/72195/exe (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72195 ceph-mds ceph NOFD /proc/72177/task/72195/fd (opendir: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72310 safe_time ceph cwd unknown /proc/72177/task/72310/cwd (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72310 safe_time ceph rtd unknown /proc/72177/task/72310/root (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72310 safe_time ceph txt unknown /proc/72177/task/72310/exe (readlink: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72177 72310 safe_time ceph NOFD /proc/72177/task/72310/fd (opendir: Permission denied) 2024-05-08T05:56:41.060 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72606 root cwd unknown /proc/72606/cwd (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72606 root rtd unknown /proc/72606/root (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72606 root txt unknown /proc/72606/exe (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:conmon 72606 root NOFD /proc/72606/fd (opendir: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72608 root cwd unknown /proc/72608/cwd (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72608 root rtd unknown /proc/72608/root (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72608 root txt unknown /proc/72608/exe (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:podman-in 72608 root NOFD /proc/72608/fd (opendir: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 ceph cwd unknown /proc/72610/cwd (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 ceph rtd unknown /proc/72610/root (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 ceph txt unknown /proc/72610/exe (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 ceph NOFD /proc/72610/fd (opendir: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72611 log ceph cwd unknown /proc/72610/task/72611/cwd (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72611 log ceph rtd unknown /proc/72610/task/72611/root (readlink: Permission denied) 2024-05-08T05:56:41.061 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72611 log ceph txt unknown /proc/72610/task/72611/exe (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72611 log ceph NOFD /proc/72610/task/72611/fd (opendir: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72613 msgr-work ceph cwd unknown /proc/72610/task/72613/cwd (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72613 msgr-work ceph rtd unknown /proc/72610/task/72613/root (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72613 msgr-work ceph txt unknown /proc/72610/task/72613/exe (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72613 msgr-work ceph NOFD /proc/72610/task/72613/fd (opendir: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72614 msgr-work ceph cwd unknown /proc/72610/task/72614/cwd (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72614 msgr-work ceph rtd unknown /proc/72610/task/72614/root (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72614 msgr-work ceph txt unknown /proc/72610/task/72614/exe (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72614 msgr-work ceph NOFD /proc/72610/task/72614/fd (opendir: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72615 msgr-work ceph cwd unknown /proc/72610/task/72615/cwd (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72615 msgr-work ceph rtd unknown /proc/72610/task/72615/root (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72615 msgr-work ceph txt unknown /proc/72610/task/72615/exe (readlink: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72615 msgr-work ceph NOFD /proc/72610/task/72615/fd (opendir: Permission denied) 2024-05-08T05:56:41.062 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72619 service ceph cwd unknown /proc/72610/task/72619/cwd (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72619 service ceph rtd unknown /proc/72610/task/72619/root (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72619 service ceph txt unknown /proc/72610/task/72619/exe (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72619 service ceph NOFD /proc/72610/task/72619/fd (opendir: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72620 admin_soc ceph cwd unknown /proc/72610/task/72620/cwd (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72620 admin_soc ceph rtd unknown /proc/72610/task/72620/root (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72620 admin_soc ceph txt unknown /proc/72610/task/72620/exe (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72620 admin_soc ceph NOFD /proc/72610/task/72620/fd (opendir: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72621 signal_ha ceph cwd unknown /proc/72610/task/72621/cwd (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72621 signal_ha ceph rtd unknown /proc/72610/task/72621/root (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72621 signal_ha ceph txt unknown /proc/72610/task/72621/exe (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72621 signal_ha ceph NOFD /proc/72610/task/72621/fd (opendir: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72622 io_contex ceph cwd unknown /proc/72610/task/72622/cwd (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72622 io_contex ceph rtd unknown /proc/72610/task/72622/root (readlink: Permission denied) 2024-05-08T05:56:41.063 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72622 io_contex ceph txt unknown /proc/72610/task/72622/exe (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72622 io_contex ceph NOFD /proc/72610/task/72622/fd (opendir: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72623 io_contex ceph cwd unknown /proc/72610/task/72623/cwd (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72623 io_contex ceph rtd unknown /proc/72610/task/72623/root (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72623 io_contex ceph txt unknown /proc/72610/task/72623/exe (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72623 io_contex ceph NOFD /proc/72610/task/72623/fd (opendir: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72624 ms_dispat ceph cwd unknown /proc/72610/task/72624/cwd (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72624 ms_dispat ceph rtd unknown /proc/72610/task/72624/root (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72624 ms_dispat ceph txt unknown /proc/72610/task/72624/exe (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72624 ms_dispat ceph NOFD /proc/72610/task/72624/fd (opendir: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72625 ms_local ceph cwd unknown /proc/72610/task/72625/cwd (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72625 ms_local ceph rtd unknown /proc/72610/task/72625/root (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72625 ms_local ceph txt unknown /proc/72610/task/72625/exe (readlink: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72625 ms_local ceph NOFD /proc/72610/task/72625/fd (opendir: Permission denied) 2024-05-08T05:56:41.064 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72626 safe_time ceph cwd unknown /proc/72610/task/72626/cwd (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72626 safe_time ceph rtd unknown /proc/72610/task/72626/root (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72626 safe_time ceph txt unknown /proc/72610/task/72626/exe (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72626 safe_time ceph NOFD /proc/72610/task/72626/fd (opendir: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72627 safe_time ceph cwd unknown /proc/72610/task/72627/cwd (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72627 safe_time ceph rtd unknown /proc/72610/task/72627/root (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72627 safe_time ceph txt unknown /proc/72610/task/72627/exe (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72627 safe_time ceph NOFD /proc/72610/task/72627/fd (opendir: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72628 ceph-mds ceph cwd unknown /proc/72610/task/72628/cwd (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72628 ceph-mds ceph rtd unknown /proc/72610/task/72628/root (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72628 ceph-mds ceph txt unknown /proc/72610/task/72628/exe (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72628 ceph-mds ceph NOFD /proc/72610/task/72628/fd (opendir: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72651 safe_time ceph cwd unknown /proc/72610/task/72651/cwd (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72651 safe_time ceph rtd unknown /proc/72610/task/72651/root (readlink: Permission denied) 2024-05-08T05:56:41.065 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72651 safe_time ceph txt unknown /proc/72610/task/72651/exe (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-mds 72610 72651 safe_time ceph NOFD /proc/72610/task/72651/fd (opendir: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 74941 root cwd unknown /proc/74941/cwd (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 74941 root rtd unknown /proc/74941/root (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 74941 root txt unknown /proc/74941/exe (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/4 74941 root NOFD /proc/74941/fd (opendir: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 76379 root cwd unknown /proc/76379/cwd (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 76379 root rtd unknown /proc/76379/root (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 76379 root txt unknown /proc/76379/exe (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 76379 root NOFD /proc/76379/fd (opendir: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 78794 root cwd unknown /proc/78794/cwd (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 78794 root rtd unknown /proc/78794/root (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 78794 root txt unknown /proc/78794/exe (readlink: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 78794 root NOFD /proc/78794/fd (opendir: Permission denied) 2024-05-08T05:56:41.066 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 78797 root cwd unknown /proc/78797/cwd (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 78797 root rtd unknown /proc/78797/root (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 78797 root txt unknown /proc/78797/exe (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/5 78797 root NOFD /proc/78797/fd (opendir: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80394 root cwd unknown /proc/80394/cwd (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80394 root rtd unknown /proc/80394/root (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80394 root txt unknown /proc/80394/exe (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80394 root NOFD /proc/80394/fd (opendir: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80489 root cwd unknown /proc/80489/cwd (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80489 root rtd unknown /proc/80489/root (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80489 root txt unknown /proc/80489/exe (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/2 80489 root NOFD /proc/80489/fd (opendir: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 84882 root cwd unknown /proc/84882/cwd (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 84882 root rtd unknown /proc/84882/root (readlink: Permission denied) 2024-05-08T05:56:41.067 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 84882 root txt unknown /proc/84882/exe (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/3 84882 root NOFD /proc/84882/fd (opendir: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 86024 root cwd unknown /proc/86024/cwd (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 86024 root rtd unknown /proc/86024/root (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 86024 root txt unknown /proc/86024/exe (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 86024 root NOFD /proc/86024/fd (opendir: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 86607 root cwd unknown /proc/86607/cwd (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 86607 root rtd unknown /proc/86607/root (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 86607 root txt unknown /proc/86607/exe (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/7 86607 root NOFD /proc/86607/fd (opendir: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87918 root cwd unknown /proc/87918/cwd (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87918 root rtd unknown /proc/87918/root (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87918 root txt unknown /proc/87918/exe (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87918 root NOFD /proc/87918/fd (opendir: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87919 root cwd unknown /proc/87919/cwd (readlink: Permission denied) 2024-05-08T05:56:41.068 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87919 root rtd unknown /proc/87919/root (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87919 root txt unknown /proc/87919/exe (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/1 87919 root NOFD /proc/87919/fd (opendir: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 88047 root cwd unknown /proc/88047/cwd (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 88047 root rtd unknown /proc/88047/root (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 88047 root txt unknown /proc/88047/exe (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/0 88047 root NOFD /proc/88047/fd (opendir: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-msgr 88450 root cwd unknown /proc/88450/cwd (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-msgr 88450 root rtd unknown /proc/88450/root (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-msgr 88450 root txt unknown /proc/88450/exe (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-msgr 88450 root NOFD /proc/88450/fd (opendir: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-watc 88451 root cwd unknown /proc/88451/cwd (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-watc 88451 root rtd unknown /proc/88451/root (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-watc 88451 root txt unknown /proc/88451/exe (readlink: Permission denied) 2024-05-08T05:56:41.069 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-watc 88451 root NOFD /proc/88451/fd (opendir: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-comp 88452 root cwd unknown /proc/88452/cwd (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-comp 88452 root rtd unknown /proc/88452/root (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-comp 88452 root txt unknown /proc/88452/exe (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:ceph-comp 88452 root NOFD /proc/88452/fd (opendir: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88839 root cwd unknown /proc/88839/cwd (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88839 root rtd unknown /proc/88839/root (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88839 root txt unknown /proc/88839/exe (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88839 root NOFD /proc/88839/fd (opendir: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88899 root cwd unknown /proc/88899/cwd (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88899 root rtd unknown /proc/88899/root (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88899 root txt unknown /proc/88899/exe (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sshd 88899 root NOFD /proc/88899/fd (opendir: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 89357 root cwd unknown /proc/89357/cwd (readlink: Permission denied) 2024-05-08T05:56:41.070 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 89357 root rtd unknown /proc/89357/root (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 89357 root txt unknown /proc/89357/exe (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:sudo 89357 root NOFD /proc/89357/fd (opendir: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:daemon-he 89389 root cwd unknown /proc/89389/cwd (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:daemon-he 89389 root rtd unknown /proc/89389/root (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:daemon-he 89389 root txt unknown /proc/89389/exe (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:daemon-he 89389 root NOFD /proc/89389/fd (opendir: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:bash 89415 root cwd unknown /proc/89415/cwd (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:bash 89415 root rtd unknown /proc/89415/root (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:bash 89415 root txt unknown /proc/89415/exe (readlink: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:bash 89415 root NOFD /proc/89415/fd (opendir: Permission denied) 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu rtd DIR 8,1 4096 2 / 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu txt REG 8,1 41144 7580 /usr/bin/timeout 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-08T05:56:41.071 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu 0r FIFO 0,13 0t0 264004 pipe 2024-05-08T05:56:41.072 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu 1w FIFO 0,13 0t0 264005 pipe 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:timeout 94102 ubuntu 2w FIFO 0,13 0t0 264006 pipe 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu rtd DIR 8,1 4096 2 / 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu txt REG 8,1 1389072 7149 /usr/bin/bash 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 2586930 16078 /usr/lib/locale/en_US.utf8/LC_COLLATE 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 54 16081 /usr/lib/locale/en_US.utf8/LC_NUMERIC 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 3284 28786 /usr/lib/locale/en_US.utf8/LC_TIME 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 286 28783 /usr/lib/locale/en_US.utf8/LC_MONETARY 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 57 26536 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 34 28836 /usr/lib/locale/en_US.utf8/LC_PAPER 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 77 16080 /usr/lib/locale/en_US.utf8/LC_NAME 2024-05-08T05:56:41.073 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 167 28778 /usr/lib/locale/en_US.utf8/LC_ADDRESS 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 59 28785 /usr/lib/locale/en_US.utf8/LC_TELEPHONE 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 195080 7147 /usr/lib64/libtinfo.so.6.2 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 23 28782 /usr/lib/locale/en_US.utf8/LC_MEASUREMENT 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu mem REG 8,1 369 28781 /usr/lib/locale/en_US.utf8/LC_IDENTIFICATION 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu 0r FIFO 0,13 0t0 264004 pipe 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu 1w FIFO 0,13 0t0 264005 pipe 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu 2w FIFO 0,13 0t0 264006 pipe 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:fsync-tes 94120 ubuntu 255r REG 8,1 207 661678 /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsync-tester.sh 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 94364 root cwd unknown /proc/94364/cwd (readlink: Permission denied) 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 94364 root rtd unknown /proc/94364/root (readlink: Permission denied) 2024-05-08T05:56:41.074 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 94364 root txt unknown /proc/94364/exe (readlink: Permission denied) 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:kworker/u 94364 root NOFD /proc/94364/fd (opendir: Permission denied) 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:sleep 94458 root cwd unknown /proc/94458/cwd (readlink: Permission denied) 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:sleep 94458 root rtd unknown /proc/94458/root (readlink: Permission denied) 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:sleep 94458 root txt unknown /proc/94458/exe (readlink: Permission denied) 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:sleep 94458 root NOFD /proc/94458/fd (opendir: Permission denied) 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu rtd DIR 8,1 4096 2 / 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 5431656 62287 /usr/lib64/libcrypto.so.3.2.1 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-05-08T05:56:41.075 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 0r FIFO 0,13 0t0 264004 pipe 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 1w FIFO 0,13 0t0 264005 pipe 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 2w FIFO 0,13 0t0 264006 pipe 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 3r DIR 0,20 0 1 /proc 2024-05-08T05:56:41.076 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 4r DIR 0,20 0 270458 /proc/94460/fd 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 5w FIFO 0,13 0t0 270463 pipe 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94460 ubuntu 6r FIFO 0,13 0t0 270464 pipe 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu cwd DIR 0,114 4 1099511627782 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu rtd DIR 8,1 4096 2 / 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu txt REG 8,1 179232 9797 /usr/bin/lsof 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 5431656 62287 /usr/lib64/libcrypto.so.3.2.1 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 346132 28859 /usr/lib/locale/C.utf8/LC_CTYPE 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 102672 7169 /usr/lib64/libz.so.1.2.11 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 70928 7116 /usr/lib64/libresolv.so.2 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 23992 7243 /usr/lib64/libkeyutils.so.1.10 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 66192 8888 /usr/lib64/libkrb5support.so.0.1 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 23872 7193 /usr/lib64/libcom_err.so.2.1 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 98992 8880 /usr/lib64/libk5crypto.so.3.1 2024-05-08T05:56:41.077 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 906256 8886 /usr/lib64/libkrb5.so.3.3 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 358448 8876 /usr/lib64/libgssapi_krb5.so.2.2 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 2592488 7107 /usr/lib64/libc.so.6 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 636848 7222 /usr/lib64/libpcre2-8.so.0.11.0 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 196224 8914 /usr/lib64/libtirpc.so.3.0.0 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 175752 7307 /usr/lib64/libselinux.so.1 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 26988 27963 /usr/lib64/gconv/gconv-modules.cache 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu mem REG 8,1 867384 11866 /usr/lib64/ld-linux-x86-64.so.2 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu 4r FIFO 0,13 0t0 270463 pipe 2024-05-08T05:56:41.078 INFO:tasks.workunit.client.0.smithi003.stdout:lsof 94461 ubuntu 7w FIFO 0,13 0t0 270464 pipe 2024-05-08T05:56:41.079 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T05:56:41.079 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T05:56:41.125 INFO:tasks.workunit:Stopping ['suites/fsync-tester.sh'] on client.0... 2024-05-08T05:56:41.125 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 -- 172.21.15.3:0/2586393452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c1078a0 msgr2=0x7f6a7c107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2586393452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c1078a0 0x7f6a7c107c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009930 tx=0x7f6a6c02a420 comp rx=0 tx=0).stop 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 -- 172.21.15.3:0/2586393452 shutdown_connections 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2586393452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a7c102250 0x7f6a7c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2586393452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a7c101890 0x7f6a7c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2586393452 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c1078a0 0x7f6a7c107c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 -- 172.21.15.3:0/2586393452 >> 172.21.15.3:0/2586393452 conn(0x7f6a7c0789d0 msgr2=0x7f6a7c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 -- 172.21.15.3:0/2586393452 shutdown_connections 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 -- 172.21.15.3:0/2586393452 wait complete. 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.283+0000 7f6a8346d640 1 Processor -- start 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 -- start start 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a7c101890 0x7f6a7c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 0x7f6a7c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a7c1078a0 0x7f6a7c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c111b20 con 0x7f6a7c102250 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1119a0 con 0x7f6a7c1078a0 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a8346d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c111ca0 con 0x7f6a7c101890 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 0x7f6a7c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 0x7f6a7c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35936/0 (socket says 172.21.15.3:35936) 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 -- 172.21.15.3:0/2384509071 learned_addr learned my addr 172.21.15.3:0/2384509071 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a81209640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a7c101890 0x7f6a7c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a7c1078a0 0x7f6a7c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 -- 172.21.15.3:0/2384509071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a7c101890 msgr2=0x7f6a7c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a7c101890 0x7f6a7c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 -- 172.21.15.3:0/2384509071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a7c1078a0 msgr2=0x7f6a7c19da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a7c1078a0 0x7f6a7c19da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 -- 172.21.15.3:0/2384509071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7c19e1c0 con 0x7f6a7c102250 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.284+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 0x7f6a7c199710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800d950 tx=0x7f6a7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.285+0000 7f6a727fc640 1 -- 172.21.15.3:0/2384509071 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78017030 con 0x7f6a7c102250 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.285+0000 7f6a727fc640 1 -- 172.21.15.3:0/2384509071 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78004020 con 0x7f6a7c102250 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.285+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7c19e4b0 con 0x7f6a7c102250 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.285+0000 7f6a727fc640 1 -- 172.21.15.3:0/2384509071 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780047a0 con 0x7f6a7c102250 2024-05-08T05:56:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.285+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7c1a5d40 con 0x7f6a7c102250 2024-05-08T05:56:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.287+0000 7f6a727fc640 1 -- 172.21.15.3:0/2384509071 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f6a78009a30 con 0x7f6a7c102250 2024-05-08T05:56:41.292 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.287+0000 7f6a86181740 1 Processor -- start 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.287+0000 7f6a86181740 1 -- start start 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a86181740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481422e10 0x5574814231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a86181740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x557481431fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a86181740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481432500 0x5574814348f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a86181740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5574813885d0 con 0x557481423730 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a86181740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557481341890 con 0x557481432500 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a86181740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557481384590 con 0x557481422e10 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81209640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x557481431fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81209640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x557481431fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35950/0 (socket says 172.21.15.3:35950) 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81209640 1 -- 172.21.15.3:0/493750691 learned_addr learned my addr 172.21.15.3:0/493750691 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481422e10 0x5574814231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a80a08640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481432500 0x5574814348f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 -- 172.21.15.3:0/493750691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481432500 msgr2=0x5574814348f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481432500 0x5574814348f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 -- 172.21.15.3:0/493750691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 msgr2=0x557481431fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x557481431fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 -- 172.21.15.3:0/493750691 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55748109a210 con 0x557481422e10 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81209640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x557481431fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a80a08640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481432500 0x5574814348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a727fc640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6a680b2950 0x7f6a680b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81209640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6a680b2950 0x7f6a680b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.288+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481422e10 0x5574814231f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009d90 tx=0x7f6a7400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90437dad3cd43a87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a727fc640 1 -- 172.21.15.3:0/2384509071 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f6a780e2410 con 0x7f6a7c102250 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a677fe640 1 -- 172.21.15.3:0/493750691 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74003970 con 0x557481422e10 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a677fe640 1 -- 172.21.15.3:0/493750691 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a74004030 con 0x557481422e10 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a677fe640 1 -- 172.21.15.3:0/493750691 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74011730 con 0x557481422e10 2024-05-08T05:56:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a86181740 1 -- 172.21.15.3:0/493750691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481422e10 msgr2=0x5574814231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a81209640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6a680b2950 0x7f6a680b4e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x557481425b00 tx=0x7f6a6c005fd0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.289+0000 7f6a86181740 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481422e10 0x5574814231f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009d90 tx=0x7f6a7400f730 comp rx=0 tx=0).stop 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 -- 172.21.15.3:0/493750691 shutdown_connections 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481432500 0x5574814348f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x557481431fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 --2- 172.21.15.3:0/493750691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481422e10 0x5574814231f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 -- 172.21.15.3:0/493750691 >> 172.21.15.3:0/493750691 conn(0x55748141f420 msgr2=0x55748141f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 -- 172.21.15.3:0/493750691 shutdown_connections 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 -- 172.21.15.3:0/493750691 wait complete. 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 -- start start 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 Processor -- start 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x55748142f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x55748142f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.290+0000 7f6a86181740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x557481435130 con 0x557481422e10 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a86181740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557481345370 con 0x557481423730 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a86181740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5574814352b0 con 0x557481432500 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x55748142f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a86181740 10 client.0 _ll_register_callbacks cb 0x557481099a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x55748142f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35956/0 (socket says 172.21.15.3:35956) 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81a0a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 -- 172.21.15.3:0/384504130 learned_addr learned my addr 172.21.15.3:0/384504130 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a80a08640 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x55748142f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 -- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 msgr2=0x55748142fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 -- 172.21.15.3:0/384504130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 msgr2=0x55748142f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x55748142f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 -- 172.21.15.3:0/384504130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557481166620 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a81209640 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x55748142f400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c031750 tx=0x7f6a6c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a33ad8884c8666 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a64ff9640 1 -- 172.21.15.3:0/384504130 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c04d030 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a64ff9640 1 -- 172.21.15.3:0/384504130 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f6a6c003800 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.291+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a74004030 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a64ff9640 1 -- 172.21.15.3:0/384504130 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a6c05a050 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 10 client.15609 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x557481440fe0 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a64ff9640 1 -- 172.21.15.3:0/384504130 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a74004030 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a64ff9640 1 -- 172.21.15.3:0/384504130 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f6a6c03d9b0 con 0x557481422e10 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 10 client.15609 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:41.297 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 20 client.15609 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814424f0 0x557481444900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 4 client.15609 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574813961e0 con 0x5574814424f0 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.292+0000 7f6a80a08640 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814424f0 0x557481444900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.293+0000 7f6a80a08640 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814424f0 0x557481444900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21653caff551fec8 server_cookie=7d22004a91fc3f48 in_seq=0 out_seq=0 2024-05-08T05:56:41.298 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.293+0000 7f6a64ff9640 10 client.15609 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.293+0000 7f6a64ff9640 1 -- 172.21.15.3:0/384504130 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574813961e0 con 0x5574814424f0 2024-05-08T05:56:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.293+0000 7f6a64ff9640 10 client.15609 handle_command_reply: tid=0 2024-05-08T05:56:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.293+0000 7f6a86181740 1 client.15609 shutdown 2024-05-08T05:56:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 msgr2=0x55748142f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.299 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x55748142f400 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c031750 tx=0x7f6a6c009260 comp rx=0 tx=0).stop 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 shutdown_connections 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814424f0 0x557481444900 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a64ff9640 0 client.15609 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x55748142f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 --2- 172.21.15.3:0/384504130 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x55748142f400 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 >> 172.21.15.3:0/384504130 conn(0x55748141f420 msgr2=0x557481425240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 shutdown_connections 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 1 -- 172.21.15.3:0/384504130 wait complete. 2024-05-08T05:56:41.300 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.294+0000 7f6a86181740 20 client.15609 trim_cache size 0 max 16384 2024-05-08T05:56:41.310 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 Processor -- start 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 -- start start 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481422e10 0x55748144a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x55748144a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1abb30 con 0x557481423730 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1ae3d0 con 0x557481422e10 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.309+0000 7f6a86181740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1ac1c0 con 0x557481432500 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x55748144a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x55748144a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35962/0 (socket says 172.21.15.3:35962) 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 -- 172.21.15.3:0/978171147 learned_addr learned my addr 172.21.15.3:0/978171147 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a81209640 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481422e10 0x55748144a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.311 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 -- 172.21.15.3:0/978171147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 msgr2=0x55748142c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 -- 172.21.15.3:0/978171147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481422e10 msgr2=0x55748144a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481422e10 0x55748144a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 -- 172.21.15.3:0/978171147 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557481423c70 con 0x557481423730 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a81209640 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481422e10 0x55748144a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a80a08640 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x55748144a750 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6a78004770 tx=0x7f6a780e2690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e38a5f99951040c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a66ffd640 1 -- 172.21.15.3:0/978171147 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780a8e90 con 0x557481423730 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a66ffd640 1 -- 172.21.15.3:0/978171147 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a780a9a80 con 0x557481423730 2024-05-08T05:56:41.312 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.310+0000 7f6a66ffd640 1 -- 172.21.15.3:0/978171147 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780041d0 con 0x557481423730 2024-05-08T05:56:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 -- 172.21.15.3:0/978171147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 msgr2=0x55748144a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.316 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x55748144a750 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6a78004770 tx=0x7f6a780e2690 comp rx=0 tx=0).stop 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 -- 172.21.15.3:0/978171147 shutdown_connections 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x55748142c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481423730 0x55748144a750 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 --2- 172.21.15.3:0/978171147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481422e10 0x55748144a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 -- 172.21.15.3:0/978171147 >> 172.21.15.3:0/978171147 conn(0x55748141f420 msgr2=0x55748141e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 -- 172.21.15.3:0/978171147 shutdown_connections 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 -- 172.21.15.3:0/978171147 wait complete. 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 -- start start 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.311+0000 7f6a86181740 1 Processor -- start 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x5574814a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x5574814a6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x5574814368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1ac890 con 0x557481422e10 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1add00 con 0x557481423730 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1acf60 con 0x557481432500 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a86181740 10 client.0 _ll_register_callbacks cb 0x7f6a6c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x5574814a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x5574814a6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:35976/0 (socket says 172.21.15.3:35976) 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 -- 172.21.15.3:0/2961727129 learned_addr learned my addr 172.21.15.3:0/2961727129 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x5574814a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81a0a640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x5574814368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 -- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 msgr2=0x5574814368d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x5574814368d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 -- 172.21.15.3:0/2961727129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 msgr2=0x5574814a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x5574814a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a81209640 1 -- 172.21.15.3:0/2961727129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557481437010 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.312+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x5574814a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a81209640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x5574814a6140 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009230 tx=0x7f6a6c031c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7ac51d225c7a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a64ff9640 1 -- 172.21.15.3:0/2961727129 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c057070 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a780a9a80 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a64ff9640 1 -- 172.21.15.3:0/2961727129 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f6a6c003a00 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a64ff9640 1 -- 172.21.15.3:0/2961727129 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a6c04d070 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 10 client.15615 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5574814ad920 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a64ff9640 1 -- 172.21.15.3:0/2961727129 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a780a9a80 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a64ff9640 1 -- 172.21.15.3:0/2961727129 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f6a6c049650 con 0x557481422e10 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 10 client.15615 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:41.319 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 20 client.15615 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814ae0f0 0x5574814b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 4 client.15615 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.313+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557481390050 con 0x5574814ae0f0 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.314+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814ae0f0 0x5574814b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.314+0000 7f6a80a08640 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814ae0f0 0x5574814b0510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90dc40f994ecd4c1 server_cookie=cff9e37bfda79baa in_seq=0 out_seq=0 2024-05-08T05:56:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.314+0000 7f6a64ff9640 10 client.15615 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a64ff9640 1 -- 172.21.15.3:0/2961727129 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557481390050 con 0x5574814ae0f0 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a64ff9640 10 client.15615 handle_command_reply: tid=0 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 client.15615 shutdown 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 msgr2=0x5574814a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x5574814a6140 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009230 tx=0x7f6a6c031c50 comp rx=0 tx=0).stop 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 shutdown_connections 2024-05-08T05:56:41.333 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5574814ae0f0 0x5574814b0510 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557481432500 0x5574814368d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a64ff9640 0 client.15615 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557481423730 0x5574814a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 --2- 172.21.15.3:0/2961727129 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x557481422e10 0x5574814a6140 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 >> 172.21.15.3:0/2961727129 conn(0x55748141f420 msgr2=0x557481430c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 shutdown_connections 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.332+0000 7f6a86181740 1 -- 172.21.15.3:0/2961727129 wait complete. 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a86181740 20 client.15615 trim_cache size 0 max 16384 2024-05-08T05:56:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:{ 2024-05-08T05:56:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "message": "", 2024-05-08T05:56:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout: "return_code": 0 2024-05-08T05:56:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stdout:} 2024-05-08T05:56:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6a680b2950 msgr2=0x7f6a680b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6a680b2950 0x7f6a680b4e10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x557481425b00 tx=0x7f6a6c005fd0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 msgr2=0x7f6a7c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 0x7f6a7c199710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800d950 tx=0x7f6a7800de10 comp rx=0 tx=0).stop 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 shutdown_connections 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f6a680b2950 0x7f6a680b4e10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6a7c1078a0 0x7f6a7c19da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f6a7c102250 0x7f6a7c199710 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 --2- 172.21.15.3:0/2384509071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a7c101890 0x7f6a7c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 >> 172.21.15.3:0/2384509071 conn(0x7f6a7c0789d0 msgr2=0x7f6a7c06a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 shutdown_connections 2024-05-08T05:56:41.336 INFO:teuthology.task.background_exec.ubuntu@smithi003.front.sepia.ceph.com.smithi003.stderr:2024-05-08T05:56:41.333+0000 7f6a8346d640 1 -- 172.21.15.3:0/2384509071 wait complete. 2024-05-08T05:56:41.420 DEBUG:teuthology.parallel:result is None 2024-05-08T05:56:41.420 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T05:56:41.451 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T05:56:41.452 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-08T05:56:41.464 INFO:tasks.quiescer:joining Quiescers 2024-05-08T05:56:41.464 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-08T05:56:41.464 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:56:41.744 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:42.153 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:41 smithi003 ceph-mon[30137]: pgmap v91: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 11 op/s 2024-05-08T05:56:42.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:41 smithi089 ceph-mon[34495]: pgmap v91: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 11 op/s 2024-05-08T05:56:42.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:41 smithi104 ceph-mon[34559]: pgmap v91: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 17 MiB/s wr, 11 op/s 2024-05-08T05:56:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 -- 172.21.15.3:0/266359029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310104130 msgr2=0x7f5310108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 --2- 172.21.15.3:0/266359029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310104130 0x7f5310108140 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f53000099c0 tx=0x7f5300009cd0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.720 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 -- 172.21.15.3:0/266359029 shutdown_connections 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 --2- 172.21.15.3:0/266359029 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310108870 0x7f53101103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 --2- 172.21.15.3:0/266359029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5310103780 0x7f5310103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 --2- 172.21.15.3:0/266359029 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310104130 0x7f5310108140 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 -- 172.21.15.3:0/266359029 >> 172.21.15.3:0/266359029 conn(0x7f531007a610 msgr2=0x7f531007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 -- 172.21.15.3:0/266359029 shutdown_connections 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 -- 172.21.15.3:0/266359029 wait complete. 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.718+0000 7f5318214700 1 Processor -- start 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 -- start start 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310103780 0x7f531019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:43.721 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5310104130 0x7f531019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 0x7f53101998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5310113650 con 0x7f5310108870 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f53101134d0 con 0x7f5310103780 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5318214700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f53101137d0 con 0x7f5310104130 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 0x7f53101998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 0x7f53101998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56916/0 (socket says 172.21.15.3:56916) 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 -- 172.21.15.3:0/2467618901 learned_addr learned my addr 172.21.15.3:0/2467618901 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53157af700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5310104130 0x7f531019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f5315fb0700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310103780 0x7f531019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 -- 172.21.15.3:0/2467618901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5310104130 msgr2=0x7f531019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5310104130 0x7f531019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.722 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 -- 172.21.15.3:0/2467618901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310103780 msgr2=0x7f531019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310103780 0x7f531019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.719+0000 7f53167b1700 1 -- 172.21.15.3:0/2467618901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531019a050 con 0x7f5310108870 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.720+0000 7f53167b1700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 0x7f53101998f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5308007ee0 tx=0x7f530800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.720+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308004bb0 con 0x7f5310108870 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.720+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531019a340 con 0x7f5310108870 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.720+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5308004d50 con 0x7f5310108870 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.720+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308005920 con 0x7f5310108870 2024-05-08T05:56:43.723 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.720+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101a6600 con 0x7f5310108870 2024-05-08T05:56:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.721+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531010ba40 con 0x7f5310108870 2024-05-08T05:56:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.722+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f530801a020 con 0x7f5310108870 2024-05-08T05:56:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.723+0000 7f5306ffd700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f52fc0b29f0 0x7f52fc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.723+0000 7f5315fb0700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f52fc0b29f0 0x7f52fc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.723+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f53080e26b0 con 0x7f5310108870 2024-05-08T05:56:43.724 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.724+0000 7f5315fb0700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f52fc0b29f0 0x7f52fc0b4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f531019aa30 tx=0x7f530c00b3c0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:43.727 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.727+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5308010070 con 0x7f5310108870 2024-05-08T05:56:43.989 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.989+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f531019ae20 con 0x7f5310108870 2024-05-08T05:56:43.990 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:43 smithi003 ceph-mon[30137]: pgmap v92: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-08T05:56:43.990 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.990+0000 7f5306ffd700 1 -- 172.21.15.3:0/2467618901 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7f53080a9d20 con 0x7f5310108870 2024-05-08T05:56:43.990 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:56:43.991 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:56:43.992 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f52fc0b29f0 msgr2=0x7f52fc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f52fc0b29f0 0x7f52fc0b4eb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f531019aa30 tx=0x7f530c00b3c0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 msgr2=0x7f53101998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 0x7f53101998f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5308007ee0 tx=0x7f530800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 shutdown_connections 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5310108870 0x7f53101998f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5310103780 0x7f531019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f52fc0b29f0 0x7f52fc0b4eb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 --2- 172.21.15.3:0/2467618901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5310104130 0x7f531019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.992+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 >> 172.21.15.3:0/2467618901 conn(0x7f531007a610 msgr2=0x7f5310101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.993+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 shutdown_connections 2024-05-08T05:56:43.993 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:43.993+0000 7f5318214700 1 -- 172.21.15.3:0/2467618901 wait complete. 2024-05-08T05:56:43.994 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T05:56:44.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:43 smithi089 ceph-mon[34495]: pgmap v92: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-08T05:56:44.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:43 smithi104 ceph-mon[34559]: pgmap v92: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 18 op/s 2024-05-08T05:56:44.813 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24493 quiesce db --cancel --all' 2024-05-08T05:56:44.814 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24493 quiesce db --cancel --all 2024-05-08T05:56:45.068 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:45.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:44 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2467618901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:45.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:44 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2467618901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:45.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:44 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2467618901' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:46.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:45 smithi003 ceph-mon[30137]: pgmap v93: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-05-08T05:56:46.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:45 smithi104 ceph-mon[34559]: pgmap v93: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-05-08T05:56:46.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:45 smithi089 ceph-mon[34495]: pgmap v93: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-05-08T05:56:47.009 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 -- 172.21.15.3:0/4169185054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089006bc70 msgr2=0x7f089006c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 --2- 172.21.15.3:0/4169185054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089006bc70 0x7f089006c0f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f08780099c0 tx=0x7f0878009cd0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 -- 172.21.15.3:0/4169185054 shutdown_connections 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 --2- 172.21.15.3:0/4169185054 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f089006c630 0x7f0890110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 --2- 172.21.15.3:0/4169185054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0890109590 0x7f0890109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 --2- 172.21.15.3:0/4169185054 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f089006bc70 0x7f089006c0f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 -- 172.21.15.3:0/4169185054 >> 172.21.15.3:0/4169185054 conn(0x7f0890079440 msgr2=0x7f0890079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 -- 172.21.15.3:0/4169185054 shutdown_connections 2024-05-08T05:56:47.010 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.009+0000 7f08950a5700 1 -- 172.21.15.3:0/4169185054 wait complete. 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 Processor -- start 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 -- start start 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089006bc70 0x7f089019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f089006c630 0x7f089019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 0x7f08901999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f0890113690 con 0x7f0890109590 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0890113510 con 0x7f089006bc70 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f08950a5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0890113810 con 0x7f089006c630 2024-05-08T05:56:47.012 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f088f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 0x7f08901999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f088ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089006bc70 0x7f089019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.010+0000 7f088f59e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 0x7f08901999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56944/0 (socket says 172.21.15.3:56944) 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 -- 172.21.15.3:0/1469538469 learned_addr learned my addr 172.21.15.3:0/1469538469 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088ed9d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089006bc70 0x7f089019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:38874/0 (socket says 172.21.15.3:38874) 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088659c700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f089006c630 0x7f089019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 -- 172.21.15.3:0/1469538469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f089006c630 msgr2=0x7f089019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f089006c630 0x7f089019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 -- 172.21.15.3:0/1469538469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089006bc70 msgr2=0x7f089019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089006bc70 0x7f089019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 -- 172.21.15.3:0/1469538469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089019a110 con 0x7f0890109590 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088f59e700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 0x7f08901999b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f088800ada0 tx=0x7f088800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.013 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088cd99700 1 -- 172.21.15.3:0/1469538469 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888017020 con 0x7f0890109590 2024-05-08T05:56:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088cd99700 1 -- 172.21.15.3:0/1469538469 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f088800cd80 con 0x7f0890109590 2024-05-08T05:56:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089019a400 con 0x7f0890109590 2024-05-08T05:56:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f088cd99700 1 -- 172.21.15.3:0/1469538469 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888005260 con 0x7f0890109590 2024-05-08T05:56:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.011+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08901a6670 con 0x7f0890109590 2024-05-08T05:56:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.014+0000 7f088cd99700 1 -- 172.21.15.3:0/1469538469 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f088800c8d0 con 0x7f0890109590 2024-05-08T05:56:47.014 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.014+0000 7f088cd99700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f087c0b29f0 0x7f087c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f087c0b29f0 0x7f087c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.015 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f088cd99700 1 -- 172.21.15.3:0/1469538469 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f08880e20f0 con 0x7f0890109590 2024-05-08T05:56:47.016 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 Processor -- start 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 -- start start 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f087c0b29f0 0x7f087c0b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0890100b00 tx=0x7f088000a380 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec937b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 0x5567ec931e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9323a0 0x5567ec93e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5567ec8839b0 con 0x5567ec9319e0 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5567ec7e8e10 con 0x5567ec937730 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.015+0000 7f089f938b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5567ec87f970 con 0x5567ec9323a0 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 0x5567ec931e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 0x5567ec931e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56956/0 (socket says 172.21.15.3:56956) 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 -- 172.21.15.3:0/1266611468 learned_addr learned my addr 172.21.15.3:0/1266611468 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec937b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088f59e700 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9323a0 0x5567ec93e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 -- 172.21.15.3:0/1266611468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9323a0 msgr2=0x5567ec93e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9323a0 0x5567ec93e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 -- 172.21.15.3:0/1266611468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 msgr2=0x5567ec937b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec937b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 -- 172.21.15.3:0/1266611468 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ec7d0940 con 0x5567ec9319e0 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088659c700 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 0x5567ec931e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f08780096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6dd02fc748265c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088559a700 1 -- 172.21.15.3:0/1266611468 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878003a60 con 0x5567ec9319e0 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088559a700 1 -- 172.21.15.3:0/1266611468 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0878026070 con 0x5567ec9319e0 2024-05-08T05:56:47.018 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.016+0000 7f088559a700 1 -- 172.21.15.3:0/1266611468 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878005460 con 0x5567ec9319e0 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 -- 172.21.15.3:0/1266611468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 msgr2=0x5567ec931e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 0x5567ec931e60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f08780096c0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 -- 172.21.15.3:0/1266611468 shutdown_connections 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9323a0 0x5567ec93e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec937b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 --2- 172.21.15.3:0/1266611468 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9319e0 0x5567ec931e60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 -- 172.21.15.3:0/1266611468 >> 172.21.15.3:0/1266611468 conn(0x5567ec85dbe0 msgr2=0x5567ec92c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 -- 172.21.15.3:0/1266611468 shutdown_connections 2024-05-08T05:56:47.022 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.017+0000 7f089f938b80 1 -- 172.21.15.3:0/1266611468 wait complete. 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 -- start start 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 Processor -- start 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec9319e0 0x5567ec93e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec93b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec937730 0x5567ec93b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x5567ec942e40 con 0x5567ec9323a0 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5567ec83cf40 con 0x5567ec9319e0 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5567ec942fc0 con 0x5567ec937730 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f089f938b80 10 client.0 _ll_register_callbacks cb 0x5567ec817620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec93b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec93b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56968/0 (socket says 172.21.15.3:56968) 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 -- 172.21.15.3:0/758619656 learned_addr learned my addr 172.21.15.3:0/758619656 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088ed9d700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec9319e0 0x5567ec93e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088f59e700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec937730 0x5567ec93b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 -- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec937730 msgr2=0x5567ec93b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec937730 0x5567ec93b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 -- 172.21.15.3:0/758619656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec9319e0 msgr2=0x5567ec93e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.018+0000 7f088659c700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec9319e0 0x5567ec93e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f088659c700 1 -- 172.21.15.3:0/758619656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ec83a7d0 con 0x5567ec9323a0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f088ed9d700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec9319e0 0x5567ec93e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f088659c700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec93b470 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f265cdb629bee06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f08767fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f08767fc700 1 -- 172.21.15.3:0/758619656 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087800bca0 con 0x5567ec9323a0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f08767fc700 1 -- 172.21.15.3:0/758619656 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f087800be40 con 0x5567ec9323a0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0878026070 con 0x5567ec9323a0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f08767fc700 1 -- 172.21.15.3:0/758619656 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f087803b050 con 0x5567ec9323a0 2024-05-08T05:56:47.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f089f938b80 10 client.15633 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5567ec94d510 con 0x5567ec9323a0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.019+0000 7f08767fc700 1 -- 172.21.15.3:0/758619656 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0878026070 con 0x5567ec9323a0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f08767fc700 1 -- 172.21.15.3:0/758619656 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f087800f980 con 0x5567ec9323a0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f089f938b80 10 client.15633 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f089f938b80 10 client.15633 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f089f938b80 20 client.15633 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f089f938b80 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec94f9e0 0x5567ec951dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f089f938b80 4 client.15633 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567ec891390 con 0x5567ec94f9e0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f088ed9d700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec94f9e0 0x5567ec951dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f088ed9d700 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec94f9e0 0x5567ec951dd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58685ac1749ca1b7 server_cookie=e095f15da5e0e69 in_seq=0 out_seq=0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.020+0000 7f08767fc700 10 client.15633 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f08767fc700 1 -- 172.21.15.3:0/758619656 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567ec891390 con 0x5567ec94f9e0 2024-05-08T05:56:47.025 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f08767fc700 10 client.15633 handle_command_reply: tid=0 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 client.15633 shutdown 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 msgr2=0x5567ec93b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec93b470 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0878009990 tx=0x7f0878003f60 comp rx=0 tx=0).stop 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 shutdown_connections 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec937730 0x5567ec93b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec94f9e0 0x5567ec951dd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec9319e0 0x5567ec93e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 --2- 172.21.15.3:0/758619656 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec93b470 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f08767fc700 0 client.15633 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 >> 172.21.15.3:0/758619656 conn(0x5567ec85dbe0 msgr2=0x5567ec936ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 shutdown_connections 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 1 -- 172.21.15.3:0/758619656 wait complete. 2024-05-08T05:56:47.026 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.021+0000 7f089f938b80 20 client.15633 trim_cache size 0 max 16384 2024-05-08T05:56:47.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.041+0000 7f089f938b80 1 Processor -- start 2024-05-08T05:56:47.041 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.041+0000 7f089f938b80 1 -- start start 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f089f938b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec926690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f089f938b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec9310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f089f938b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec936500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f089f938b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08901ab090 con 0x5567ec9323a0 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f089f938b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08901ad930 con 0x5567ec937730 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f089f938b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08901ab720 con 0x5567ec9319e0 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec9310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec9310e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56982/0 (socket says 172.21.15.3:56982) 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 -- 172.21.15.3:0/1524399776 learned_addr learned my addr 172.21.15.3:0/1524399776 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088f59e700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec936500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec926690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.044 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 -- 172.21.15.3:0/1524399776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 msgr2=0x5567ec926690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec926690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 -- 172.21.15.3:0/1524399776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 msgr2=0x5567ec936500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec936500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 -- 172.21.15.3:0/1524399776 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567ec9393d0 con 0x5567ec9323a0 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec926690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088f59e700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec936500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.042+0000 7f088659c700 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec9310e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f087800ff70 tx=0x7f08780037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5071293928cf6e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f0884d99700 1 -- 172.21.15.3:0/1524399776 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780058e0 con 0x5567ec9323a0 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f0884d99700 1 -- 172.21.15.3:0/1524399776 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0878026040 con 0x5567ec9323a0 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f0884d99700 1 -- 172.21.15.3:0/1524399776 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878032a30 con 0x5567ec9323a0 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 -- 172.21.15.3:0/1524399776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 msgr2=0x5567ec9310e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec9310e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f087800ff70 tx=0x7f08780037c0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 -- 172.21.15.3:0/1524399776 shutdown_connections 2024-05-08T05:56:47.045 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec936500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec926690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 --2- 172.21.15.3:0/1524399776 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec9310e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 -- 172.21.15.3:0/1524399776 >> 172.21.15.3:0/1524399776 conn(0x5567ec85dbe0 msgr2=0x5567ec9309d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 -- 172.21.15.3:0/1524399776 shutdown_connections 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.043+0000 7f089f938b80 1 -- 172.21.15.3:0/1524399776 wait complete. 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 -- start start 2024-05-08T05:56:47.046 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 Processor -- start 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec954fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec955520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec953590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f08901abdf0 con 0x5567ec9323a0 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f08901ad260 con 0x5567ec937730 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08901ac4c0 con 0x5567ec9319e0 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f089f938b80 10 client.0 _ll_register_callbacks cb 0x5567ec817620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:56:47.049 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec955520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec955520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:56988/0 (socket says 172.21.15.3:56988) 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 -- 172.21.15.3:0/1260267835 learned_addr learned my addr 172.21.15.3:0/1260267835 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec954fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088f59e700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec953590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 -- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 msgr2=0x5567ec954fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec954fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 -- 172.21.15.3:0/1260267835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 msgr2=0x5567ec953590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec953590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088659c700 1 -- 172.21.15.3:0/1260267835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567ec953dd0 con 0x5567ec9323a0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.044+0000 7f088f59e700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec953590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f088659c700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec955520 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f087800fec0 tx=0x7f0878005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751cf3f62b37277b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f08767fc700 10 client.0 ms_handle_connect on v2:172.21.15.3:3300/0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f08767fc700 1 -- 172.21.15.3:0/1260267835 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087800baa0 con 0x5567ec9323a0 2024-05-08T05:56:47.050 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f08767fc700 1 -- 172.21.15.3:0/1260267835 <== mon.0 v2:172.21.15.3:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0878032400 con 0x5567ec9323a0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0878026040 con 0x5567ec9323a0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f08767fc700 1 -- 172.21.15.3:0/1260267835 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f087804f050 con 0x5567ec9323a0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f089f938b80 10 client.15639 fetch_fsmap learned FSMap version 23 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5567ec9bab10 con 0x5567ec9323a0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f08767fc700 1 -- 172.21.15.3:0/1260267835 <== mon.0 v2:172.21.15.3:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0878026040 con 0x5567ec9323a0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f08767fc700 1 -- 172.21.15.3:0/1260267835 <== mon.0 v2:172.21.15.3:3300/0 5 ==== fsmap(e 23) ==== 8330+0+0 (secure 0 0 0) 0x7f087800f460 con 0x5567ec9323a0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f089f938b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 23 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.045+0000 7f089f938b80 10 client.15639 resolve_mds: validated gid 24493 aka daemon mds.i 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f089f938b80 20 client.15639 populate_metadata read hostname 'smithi003' 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f089f938b80 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec9bb020 0x5567ec9bd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f089f938b80 4 client.15639 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 --> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5567ec88b430 con 0x5567ec9bb020 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec9bb020 0x5567ec9bd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f08767fc700 10 client.15639 ms_handle_connect on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:47.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f088ed9d700 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec9bb020 0x5567ec9bd440 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fdea5e02541cde server_cookie=48844d24a9b94b08 in_seq=0 out_seq=0 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f08767fc700 1 -- 172.21.15.3:0/1260267835 <== mds.0 v2:172.21.15.104:6838/3320987620 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x5567ec88b430 con 0x5567ec9bb020 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f08767fc700 10 client.15639 handle_command_reply: tid=0 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.046+0000 7f089f938b80 1 client.15639 shutdown 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 msgr2=0x5567ec955520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec955520 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f087800fec0 tx=0x7f0878005d30 comp rx=0 tx=0).stop 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 shutdown_connections 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:6838/3320987620,v1:172.21.15.104:6839/3320987620] conn(0x5567ec9bb020 0x5567ec9bd440 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5567ec937730 0x5567ec953590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5567ec9319e0 0x5567ec954fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 --2- 172.21.15.3:0/1260267835 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x5567ec9323a0 0x5567ec955520 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 >> 172.21.15.3:0/1260267835 conn(0x5567ec85dbe0 msgr2=0x5567ec926510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08767fc700 0 client.15639 ms_handle_reset on v2:172.21.15.104:6838/3320987620 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 shutdown_connections 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 1 -- 172.21.15.3:0/1260267835 wait complete. 2024-05-08T05:56:47.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f089f938b80 20 client.15639 trim_cache size 0 max 16384 2024-05-08T05:56:47.053 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T05:56:47.053 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 23, 2024-05-08T05:56:47.053 INFO:teuthology.orchestra.run.smithi003.stdout: "leader": 24493, 2024-05-08T05:56:47.053 INFO:teuthology.orchestra.run.smithi003.stdout: "set_version": 8, 2024-05-08T05:56:47.053 INFO:teuthology.orchestra.run.smithi003.stdout: "sets": {} 2024-05-08T05:56:47.053 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f087c0b29f0 msgr2=0x7f087c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f087c0b29f0 0x7f087c0b4eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0890100b00 tx=0x7f088000a380 comp rx=0 tx=0).stop 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 msgr2=0x7f08901999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 0x7f08901999b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f088800ada0 tx=0x7f088800c330 comp rx=0 tx=0).stop 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 shutdown_connections 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f0890109590 0x7f08901999b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f089006bc70 0x7f089019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f087c0b29f0 0x7f087c0b4eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 --2- 172.21.15.3:0/1469538469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f089006c630 0x7f089019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:47.054 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.047+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 >> 172.21.15.3:0/1469538469 conn(0x7f0890079440 msgr2=0x7f0890100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.048+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 shutdown_connections 2024-05-08T05:56:47.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:47.048+0000 7f08950a5700 1 -- 172.21.15.3:0/1469538469 wait complete. 2024-05-08T05:56:47.444 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-08T05:56:47.444 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-05-08T05:56:47.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:56:48.139 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:56:48.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:47 smithi003 ceph-mon[30137]: pgmap v94: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-05-08T05:56:48.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:47 smithi089 ceph-mon[34495]: pgmap v94: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-05-08T05:56:48.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:47 smithi104 ceph-mon[34559]: pgmap v94: 129 pgs: 129 active+clean; 483 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 MiB/s wr, 12 op/s 2024-05-08T05:56:49.920 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 -- 172.21.15.3:0/797626188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c103780 msgr2=0x7f1c4c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/797626188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c103780 0x7f1c4c103b60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b0d0 tx=0x7f1c3c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 -- 172.21.15.3:0/797626188 shutdown_connections 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/797626188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c4c108870 0x7f1c4c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/797626188 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c103780 0x7f1c4c103b60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/797626188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c4c104130 0x7f1c4c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 -- 172.21.15.3:0/797626188 >> 172.21.15.3:0/797626188 conn(0x7f1c4c07a610 msgr2=0x7f1c4c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 -- 172.21.15.3:0/797626188 shutdown_connections 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.914+0000 7f1c51e2f700 1 -- 172.21.15.3:0/797626188 wait complete. 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.915+0000 7f1c51e2f700 1 Processor -- start 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.915+0000 7f1c51e2f700 1 -- start start 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.915+0000 7f1c51e2f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c4c103780 0x7f1c4c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.915+0000 7f1c51e2f700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c4c104130 0x7f1c4c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.921 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.915+0000 7f1c51e2f700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 0x7f1c4c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.915+0000 7f1c51e2f700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c113660 con 0x7f1c4c108870 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c51e2f700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1134e0 con 0x7f1c4c104130 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c51e2f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c1137e0 con 0x7f1c4c103780 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 0x7f1c4c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 0x7f1c4c076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:57010/0 (socket says 172.21.15.3:57010) 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4b7fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c4c103780 0x7f1c4c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 -- 172.21.15.3:0/9713793 learned_addr learned my addr 172.21.15.3:0/9713793 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4affd700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c4c104130 0x7f1c4c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 -- 172.21.15.3:0/9713793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c4c103780 msgr2=0x7f1c4c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c4c103780 0x7f1c4c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 -- 172.21.15.3:0/9713793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c4c104130 msgr2=0x7f1c4c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:49.922 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c4c104130 0x7f1c4c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 -- 172.21.15.3:0/9713793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c1a8010 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c4bfff700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 0x7f1c4c076220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c970 tx=0x7f1c4400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c44004a30 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c1a8300 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c440054d0 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400ec20 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.916+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a8840 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.918+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4c10bab0 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.919+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f1c44005000 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.920+0000 7f1c48ff9700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1c340b29f0 0x7f1c340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.920+0000 7f1c4b7fe700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1c340b29f0 0x7f1c340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.920+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1c440e23c0 con 0x7f1c4c108870 2024-05-08T05:56:49.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.920+0000 7f1c4b7fe700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1c340b29f0 0x7f1c340b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009fd0 tx=0x7f1c3c00b040 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:56:49.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:49.925+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c44014030 con 0x7f1c4c108870 2024-05-08T05:56:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:49 smithi003 ceph-mon[30137]: pgmap v95: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-05-08T05:56:50.187 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.187+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c4c10b490 con 0x7f1c4c108870 2024-05-08T05:56:50.188 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.188+0000 7f1c48ff9700 1 -- 172.21.15.3:0/9713793 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7f1c440052f0 con 0x7f1c4c108870 2024-05-08T05:56:50.188 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T05:56:50.189 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1c340b29f0 msgr2=0x7f1c340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1c340b29f0 0x7f1c340b4eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009fd0 tx=0x7f1c3c00b040 comp rx=0 tx=0).stop 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 msgr2=0x7f1c4c076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 0x7f1c4c076220 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400c970 tx=0x7f1c4400cc80 comp rx=0 tx=0).stop 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 shutdown_connections 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f1c4c108870 0x7f1c4c076220 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f1c340b29f0 0x7f1c340b4eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1c4c103780 0x7f1c4c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 --2- 172.21.15.3:0/9713793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1c4c104130 0x7f1c4c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 >> 172.21.15.3:0/9713793 conn(0x7f1c4c07a610 msgr2=0x7f1c4c101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:56:50.191 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 shutdown_connections 2024-05-08T05:56:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T05:56:50.191+0000 7f1c51e2f700 1 -- 172.21.15.3:0/9713793 wait complete. 2024-05-08T05:56:50.192 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T05:56:50.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:49 smithi089 ceph-mon[34495]: pgmap v95: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-05-08T05:56:50.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:49 smithi104 ceph-mon[34559]: pgmap v95: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 MiB/s wr, 15 op/s 2024-05-08T05:56:51.084 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/9713793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:51.119 INFO:tasks.quiescer:done joining Quiescers 2024-05-08T05:56:51.119 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-08T05:56:51.130 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:56:51.169 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-08T05:56:51.180 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-08T05:56:51.180 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:56:51.180 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T05:56:51.180 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-08T05:56:51.243 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-08T05:56:51.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/9713793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:51.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/9713793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:56:52.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:51 smithi003 ceph-mon[30137]: pgmap v96: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 KiB/s wr, 8 op/s 2024-05-08T05:56:52.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:51 smithi089 ceph-mon[34495]: pgmap v96: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 KiB/s wr, 8 op/s 2024-05-08T05:56:52.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:51 smithi104 ceph-mon[34559]: pgmap v96: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 270 KiB/s wr, 8 op/s 2024-05-08T05:56:54.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:53 smithi003 ceph-mon[30137]: pgmap v97: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 272 KiB/s wr, 10 op/s 2024-05-08T05:56:54.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:53 smithi089 ceph-mon[34495]: pgmap v97: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 272 KiB/s wr, 10 op/s 2024-05-08T05:56:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:53 smithi104 ceph-mon[34559]: pgmap v97: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 272 KiB/s wr, 10 op/s 2024-05-08T05:56:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:55 smithi003 ceph-mon[30137]: pgmap v98: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-05-08T05:56:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:56:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:55 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:56:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:55 smithi089 ceph-mon[34495]: pgmap v98: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-05-08T05:56:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:56.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:56:56.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:55 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:56:56.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:55 smithi104 ceph-mon[34559]: pgmap v98: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-05-08T05:56:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:56:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:56:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:56:56.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:55 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:56:58.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:57 smithi003 ceph-mon[30137]: pgmap v99: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-05-08T05:56:58.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:57 smithi089 ceph-mon[34495]: pgmap v99: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-05-08T05:56:58.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:57 smithi104 ceph-mon[34559]: pgmap v99: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 3 op/s 2024-05-08T05:57:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:56:59 smithi003 ceph-mon[30137]: pgmap v100: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-05-08T05:57:00.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:56:59 smithi089 ceph-mon[34495]: pgmap v100: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-05-08T05:57:00.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:56:59 smithi104 ceph-mon[34559]: pgmap v100: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-05-08T05:57:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:01 smithi003 ceph-mon[30137]: pgmap v101: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 2 op/s 2024-05-08T05:57:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:01 smithi089 ceph-mon[34495]: pgmap v101: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 2 op/s 2024-05-08T05:57:02.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:01 smithi104 ceph-mon[34559]: pgmap v101: 129 pgs: 129 active+clean; 482 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.8 KiB/s wr, 2 op/s 2024-05-08T05:57:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:03 smithi003 ceph-mon[30137]: pgmap v102: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 21 op/s 2024-05-08T05:57:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:03 smithi089 ceph-mon[34495]: pgmap v102: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 21 op/s 2024-05-08T05:57:04.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:03 smithi104 ceph-mon[34559]: pgmap v102: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 15 MiB/s wr, 21 op/s 2024-05-08T05:57:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:05 smithi003 ceph-mon[30137]: pgmap v103: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-08T05:57:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:05 smithi089 ceph-mon[34495]: pgmap v103: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-08T05:57:06.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:05 smithi104 ceph-mon[34559]: pgmap v103: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-08T05:57:08.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:07 smithi003 ceph-mon[30137]: pgmap v104: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-08T05:57:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:07 smithi089 ceph-mon[34495]: pgmap v104: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-08T05:57:08.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:07 smithi104 ceph-mon[34559]: pgmap v104: 129 pgs: 129 active+clean; 516 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 15 MiB/s wr, 20 op/s 2024-05-08T05:57:09.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:08 smithi003 ceph-mon[30137]: pgmap v105: 129 pgs: 129 active+clean; 516 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-08T05:57:09.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:08 smithi089 ceph-mon[34495]: pgmap v105: 129 pgs: 129 active+clean; 516 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-08T05:57:09.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:08 smithi104 ceph-mon[34559]: pgmap v105: 129 pgs: 129 active+clean; 516 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-08T05:57:11.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:11 smithi003 ceph-mon[30137]: pgmap v106: 129 pgs: 129 active+clean; 516 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-08T05:57:12.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:11 smithi089 ceph-mon[34495]: pgmap v106: 129 pgs: 129 active+clean; 516 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-08T05:57:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:11 smithi104 ceph-mon[34559]: pgmap v106: 129 pgs: 129 active+clean; 516 MiB data, 664 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 21 MiB/s wr, 22 op/s 2024-05-08T05:57:13.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:13 smithi003 ceph-mon[30137]: pgmap v107: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 28 op/s 2024-05-08T05:57:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:13 smithi089 ceph-mon[34495]: pgmap v107: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 28 op/s 2024-05-08T05:57:14.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:13 smithi104 ceph-mon[34559]: pgmap v107: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 21 MiB/s wr, 28 op/s 2024-05-08T05:57:15.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:15 smithi003 ceph-mon[30137]: pgmap v108: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-05-08T05:57:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:15 smithi089 ceph-mon[34495]: pgmap v108: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-05-08T05:57:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:15 smithi104 ceph-mon[34559]: pgmap v108: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-05-08T05:57:17.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:17 smithi003 ceph-mon[30137]: pgmap v109: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-05-08T05:57:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:17 smithi089 ceph-mon[34495]: pgmap v109: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-05-08T05:57:18.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:17 smithi104 ceph-mon[34559]: pgmap v109: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 6.3 MiB/s wr, 9 op/s 2024-05-08T05:57:19.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:19 smithi003 ceph-mon[30137]: pgmap v110: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-05-08T05:57:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:19 smithi089 ceph-mon[34495]: pgmap v110: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-05-08T05:57:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:19 smithi104 ceph-mon[34559]: pgmap v110: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 6.3 MiB/s wr, 10 op/s 2024-05-08T05:57:21.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:21 smithi003 ceph-mon[30137]: pgmap v111: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:57:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:21 smithi089 ceph-mon[34495]: pgmap v111: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:57:22.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:21 smithi104 ceph-mon[34559]: pgmap v111: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 7 op/s 2024-05-08T05:57:23.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:23 smithi003 ceph-mon[30137]: pgmap v112: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T05:57:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:23 smithi089 ceph-mon[34495]: pgmap v112: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T05:57:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:23 smithi104 ceph-mon[34559]: pgmap v112: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-08T05:57:25.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:25 smithi003 ceph-mon[30137]: pgmap v113: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:57:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:25 smithi089 ceph-mon[34495]: pgmap v113: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:57:26.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:25 smithi104 ceph-mon[34559]: pgmap v113: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:57:27.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:27 smithi003 ceph-mon[30137]: pgmap v114: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:57:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:27 smithi089 ceph-mon[34495]: pgmap v114: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:57:28.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:27 smithi104 ceph-mon[34559]: pgmap v114: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:57:29.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:29 smithi003 ceph-mon[30137]: pgmap v115: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:57:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:29 smithi089 ceph-mon[34495]: pgmap v115: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:57:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:29 smithi104 ceph-mon[34559]: pgmap v115: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:57:31.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:31 smithi003 ceph-mon[30137]: pgmap v116: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:31 smithi089 ceph-mon[34495]: pgmap v116: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:32.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:31 smithi104 ceph-mon[34559]: pgmap v116: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:33 smithi089 ceph-mon[34495]: pgmap v117: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:33 smithi104 ceph-mon[34559]: pgmap v117: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:33 smithi003 ceph-mon[30137]: pgmap v117: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:35.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:35 smithi003 ceph-mon[30137]: pgmap v118: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:35.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:35 smithi089 ceph-mon[34495]: pgmap v118: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:35.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:35 smithi104 ceph-mon[34559]: pgmap v118: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:37.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:37 smithi003 ceph-mon[30137]: pgmap v119: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:37 smithi089 ceph-mon[34495]: pgmap v119: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:38.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:37 smithi104 ceph-mon[34559]: pgmap v119: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:39.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:39 smithi003 ceph-mon[30137]: pgmap v120: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:39 smithi089 ceph-mon[34495]: pgmap v120: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:39 smithi104 ceph-mon[34559]: pgmap v120: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:41.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:41 smithi003 ceph-mon[30137]: pgmap v121: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:42.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:41 smithi089 ceph-mon[34495]: pgmap v121: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:41 smithi104 ceph-mon[34559]: pgmap v121: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:43.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:43 smithi003 ceph-mon[30137]: pgmap v122: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:43 smithi089 ceph-mon[34495]: pgmap v122: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:43 smithi104 ceph-mon[34559]: pgmap v122: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:45.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:45 smithi003 ceph-mon[30137]: pgmap v123: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:45 smithi089 ceph-mon[34495]: pgmap v123: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:46.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:45 smithi104 ceph-mon[34559]: pgmap v123: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:47.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:47 smithi003 ceph-mon[30137]: pgmap v124: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:48.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:47 smithi089 ceph-mon[34495]: pgmap v124: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:48.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:47 smithi104 ceph-mon[34559]: pgmap v124: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:49.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:49 smithi003 ceph-mon[30137]: pgmap v125: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:49 smithi089 ceph-mon[34495]: pgmap v125: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:49 smithi104 ceph-mon[34559]: pgmap v125: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:51.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:51 smithi003 ceph-mon[30137]: pgmap v126: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:52.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:51 smithi089 ceph-mon[34495]: pgmap v126: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:52.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:51 smithi104 ceph-mon[34559]: pgmap v126: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:53 smithi089 ceph-mon[34495]: pgmap v127: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:53 smithi104 ceph-mon[34559]: pgmap v127: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:54.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:53 smithi003 ceph-mon[30137]: pgmap v127: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:57:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:55 smithi089 ceph-mon[34495]: pgmap v128: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:56.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:55 smithi104 ceph-mon[34559]: pgmap v128: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:56.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:55 smithi003 ceph-mon[30137]: pgmap v128: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:56 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:56 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:56 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:57.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:56 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:57:57.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:56 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:56 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:56 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:57.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:56 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:57:57.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:56 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:57:57.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:56 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:57:57.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:56 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:57:57.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:56 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:57:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:57 smithi089 ceph-mon[34495]: pgmap v129: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:58.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:57 smithi104 ceph-mon[34559]: pgmap v129: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:57:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:57 smithi003 ceph-mon[30137]: pgmap v129: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:57:59 smithi089 ceph-mon[34495]: pgmap v130: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-08T05:58:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:57:59 smithi104 ceph-mon[34559]: pgmap v130: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-08T05:58:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:57:59 smithi003 ceph-mon[30137]: pgmap v130: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-05-08T05:58:02.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:01 smithi089 ceph-mon[34495]: pgmap v131: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-08T05:58:02.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:01 smithi104 ceph-mon[34559]: pgmap v131: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-08T05:58:02.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:01 smithi003 ceph-mon[30137]: pgmap v131: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-05-08T05:58:04.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:03 smithi089 ceph-mon[34495]: pgmap v132: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:04.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:03 smithi104 ceph-mon[34559]: pgmap v132: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:03 smithi003 ceph-mon[30137]: pgmap v132: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:06.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:05 smithi089 ceph-mon[34495]: pgmap v133: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:06.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:05 smithi104 ceph-mon[34559]: pgmap v133: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:06.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:05 smithi003 ceph-mon[30137]: pgmap v133: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:08.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:07 smithi089 ceph-mon[34495]: pgmap v134: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:08.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:07 smithi104 ceph-mon[34559]: pgmap v134: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:08.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:07 smithi003 ceph-mon[30137]: pgmap v134: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:10.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:09 smithi089 ceph-mon[34495]: pgmap v135: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:10.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:09 smithi104 ceph-mon[34559]: pgmap v135: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:09 smithi003 ceph-mon[30137]: pgmap v135: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:12.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:11 smithi089 ceph-mon[34495]: pgmap v136: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:12.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:11 smithi104 ceph-mon[34559]: pgmap v136: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:12.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:11 smithi003 ceph-mon[30137]: pgmap v136: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:58:14.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:13 smithi089 ceph-mon[34495]: pgmap v137: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:14.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:13 smithi104 ceph-mon[34559]: pgmap v137: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:13 smithi003 ceph-mon[30137]: pgmap v137: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:58:16.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:15 smithi089 ceph-mon[34495]: pgmap v138: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:58:16.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:15 smithi104 ceph-mon[34559]: pgmap v138: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:58:16.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:15 smithi003 ceph-mon[30137]: pgmap v138: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:58:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:17 smithi089 ceph-mon[34495]: pgmap v139: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:58:18.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:17 smithi104 ceph-mon[34559]: pgmap v139: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:58:18.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:17 smithi003 ceph-mon[30137]: pgmap v139: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:58:20.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:19 smithi089 ceph-mon[34495]: pgmap v140: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-08T05:58:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:19 smithi104 ceph-mon[34559]: pgmap v140: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-08T05:58:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:19 smithi003 ceph-mon[30137]: pgmap v140: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-08T05:58:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:21 smithi089 ceph-mon[34495]: pgmap v141: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:58:22.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:21 smithi104 ceph-mon[34559]: pgmap v141: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:58:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:21 smithi003 ceph-mon[30137]: pgmap v141: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:58:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:23 smithi089 ceph-mon[34495]: pgmap v142: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:58:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:23 smithi104 ceph-mon[34559]: pgmap v142: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:58:24.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:23 smithi003 ceph-mon[30137]: pgmap v142: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:58:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:25 smithi089 ceph-mon[34495]: pgmap v143: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:25 smithi104 ceph-mon[34559]: pgmap v143: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:25 smithi003 ceph-mon[30137]: pgmap v143: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:27.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:27 smithi003 ceph-mon[30137]: pgmap v144: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:27 smithi089 ceph-mon[34495]: pgmap v144: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:28.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:27 smithi104 ceph-mon[34559]: pgmap v144: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:29 smithi089 ceph-mon[34495]: pgmap v145: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:29 smithi104 ceph-mon[34559]: pgmap v145: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:30.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:29 smithi003 ceph-mon[30137]: pgmap v145: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:31 smithi089 ceph-mon[34495]: pgmap v146: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:32.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:31 smithi104 ceph-mon[34559]: pgmap v146: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:32.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:31 smithi003 ceph-mon[30137]: pgmap v146: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:33 smithi089 ceph-mon[34495]: pgmap v147: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:33 smithi104 ceph-mon[34559]: pgmap v147: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:33 smithi003 ceph-mon[30137]: pgmap v147: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:35 smithi089 ceph-mon[34495]: pgmap v148: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:36.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:35 smithi104 ceph-mon[34559]: pgmap v148: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:36.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:35 smithi003 ceph-mon[30137]: pgmap v148: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:37 smithi089 ceph-mon[34495]: pgmap v149: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:37 smithi104 ceph-mon[34559]: pgmap v149: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:38.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:37 smithi003 ceph-mon[30137]: pgmap v149: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:39 smithi089 ceph-mon[34495]: pgmap v150: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:39 smithi104 ceph-mon[34559]: pgmap v150: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:39 smithi003 ceph-mon[30137]: pgmap v150: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:41 smithi089 ceph-mon[34495]: pgmap v151: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:41 smithi104 ceph-mon[34559]: pgmap v151: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:42.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:41 smithi003 ceph-mon[30137]: pgmap v151: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:43 smithi089 ceph-mon[34495]: pgmap v152: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:44.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:43 smithi104 ceph-mon[34559]: pgmap v152: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:44.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:43 smithi003 ceph-mon[30137]: pgmap v152: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:45 smithi089 ceph-mon[34495]: pgmap v153: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:46.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:45 smithi104 ceph-mon[34559]: pgmap v153: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:46.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:45 smithi003 ceph-mon[30137]: pgmap v153: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:48.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:47 smithi089 ceph-mon[34495]: pgmap v154: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:48.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:47 smithi104 ceph-mon[34559]: pgmap v154: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:48.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:47 smithi003 ceph-mon[30137]: pgmap v154: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:49 smithi089 ceph-mon[34495]: pgmap v155: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:49 smithi104 ceph-mon[34559]: pgmap v155: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:50.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:49 smithi003 ceph-mon[30137]: pgmap v155: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:52.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:51 smithi089 ceph-mon[34495]: pgmap v156: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:52.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:51 smithi104 ceph-mon[34559]: pgmap v156: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:52.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:51 smithi003 ceph-mon[30137]: pgmap v156: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:53 smithi089 ceph-mon[34495]: pgmap v157: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:54.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:53 smithi104 ceph-mon[34559]: pgmap v157: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:54.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:53 smithi003 ceph-mon[30137]: pgmap v157: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:58:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:55 smithi089 ceph-mon[34495]: pgmap v158: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:56.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:55 smithi104 ceph-mon[34559]: pgmap v158: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:55 smithi003 ceph-mon[30137]: pgmap v158: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:57.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:56 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:57.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:56 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:57.061 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:56 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:58:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:57 smithi089 ceph-mon[34495]: pgmap v159: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:57 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:57 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:58.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:57 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:58:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:57 smithi104 ceph-mon[34559]: pgmap v159: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:57 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:57 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:57 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:58:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:57 smithi003 ceph-mon[30137]: pgmap v159: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:58:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:57 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:58:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:57 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:58:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:57 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:59:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:58:59 smithi104 ceph-mon[34559]: pgmap v160: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:00.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:58:59 smithi003 ceph-mon[30137]: pgmap v160: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:00.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:58:59 smithi089 ceph-mon[34495]: pgmap v160: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:02.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:01 smithi104 ceph-mon[34559]: pgmap v161: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:02.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:01 smithi003 ceph-mon[30137]: pgmap v161: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:01 smithi089 ceph-mon[34495]: pgmap v161: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:03 smithi003 ceph-mon[30137]: pgmap v162: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:03 smithi089 ceph-mon[34495]: pgmap v162: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:04.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:03 smithi104 ceph-mon[34559]: pgmap v162: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:05 smithi003 ceph-mon[30137]: pgmap v163: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:05 smithi089 ceph-mon[34495]: pgmap v163: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:05 smithi104 ceph-mon[34559]: pgmap v163: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:08.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:07 smithi003 ceph-mon[30137]: pgmap v164: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:07 smithi089 ceph-mon[34495]: pgmap v164: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:08.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:07 smithi104 ceph-mon[34559]: pgmap v164: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:09 smithi003 ceph-mon[30137]: pgmap v165: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:09 smithi089 ceph-mon[34495]: pgmap v165: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:09 smithi104 ceph-mon[34559]: pgmap v165: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:12.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:11 smithi003 ceph-mon[30137]: pgmap v166: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:12.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:11 smithi089 ceph-mon[34495]: pgmap v166: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:12.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:11 smithi104 ceph-mon[34559]: pgmap v166: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-05-08T05:59:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:13 smithi003 ceph-mon[30137]: pgmap v167: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:14.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:13 smithi089 ceph-mon[34495]: pgmap v167: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:13 smithi104 ceph-mon[34559]: pgmap v167: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-05-08T05:59:16.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:15 smithi003 ceph-mon[30137]: pgmap v168: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:59:16.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:15 smithi089 ceph-mon[34495]: pgmap v168: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:59:16.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:15 smithi104 ceph-mon[34559]: pgmap v168: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:59:18.073 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:17 smithi104 ceph-mon[34559]: pgmap v169: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:59:18.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:17 smithi003 ceph-mon[30137]: pgmap v169: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:59:18.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:17 smithi089 ceph-mon[34495]: pgmap v169: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-05-08T05:59:20.065 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:19 smithi089 ceph-mon[34495]: pgmap v170: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-08T05:59:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:19 smithi003 ceph-mon[30137]: pgmap v170: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-08T05:59:20.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:19 smithi104 ceph-mon[34559]: pgmap v170: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-08T05:59:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:21 smithi003 ceph-mon[30137]: pgmap v171: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:59:22.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:21 smithi089 ceph-mon[34495]: pgmap v171: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:59:22.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:21 smithi104 ceph-mon[34559]: pgmap v171: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T05:59:24.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:23 smithi003 ceph-mon[30137]: pgmap v172: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:59:24.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:23 smithi089 ceph-mon[34495]: pgmap v172: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:59:24.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:23 smithi104 ceph-mon[34559]: pgmap v172: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T05:59:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:25 smithi003 ceph-mon[30137]: pgmap v173: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:26.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:25 smithi089 ceph-mon[34495]: pgmap v173: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:26.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:25 smithi104 ceph-mon[34559]: pgmap v173: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:28.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:27 smithi003 ceph-mon[30137]: pgmap v174: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:28.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:27 smithi089 ceph-mon[34495]: pgmap v174: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:28.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:27 smithi104 ceph-mon[34559]: pgmap v174: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:30.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:29 smithi003 ceph-mon[30137]: pgmap v175: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:30.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:29 smithi089 ceph-mon[34495]: pgmap v175: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:30.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:29 smithi104 ceph-mon[34559]: pgmap v175: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:31 smithi089 ceph-mon[34495]: pgmap v176: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:32.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:31 smithi003 ceph-mon[30137]: pgmap v176: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:32.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:31 smithi104 ceph-mon[34559]: pgmap v176: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:33 smithi003 ceph-mon[30137]: pgmap v177: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:34.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:33 smithi089 ceph-mon[34495]: pgmap v177: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:34.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:33 smithi104 ceph-mon[34559]: pgmap v177: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:36.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:35 smithi003 ceph-mon[30137]: pgmap v178: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:36.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:35 smithi089 ceph-mon[34495]: pgmap v178: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:36.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:35 smithi104 ceph-mon[34559]: pgmap v178: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:38.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:37 smithi003 ceph-mon[30137]: pgmap v179: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:38.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:37 smithi089 ceph-mon[34495]: pgmap v179: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:38.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:37 smithi104 ceph-mon[34559]: pgmap v179: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:39 smithi003 ceph-mon[30137]: pgmap v180: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:40.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:39 smithi089 ceph-mon[34495]: pgmap v180: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:40.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:39 smithi104 ceph-mon[34559]: pgmap v180: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:42.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:41 smithi003 ceph-mon[30137]: pgmap v181: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:42.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:41 smithi089 ceph-mon[34495]: pgmap v181: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:41 smithi104 ceph-mon[34559]: pgmap v181: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:44.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:43 smithi003 ceph-mon[30137]: pgmap v182: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:44.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:43 smithi089 ceph-mon[34495]: pgmap v182: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:44.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:43 smithi104 ceph-mon[34559]: pgmap v182: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:46.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:45 smithi003 ceph-mon[30137]: pgmap v183: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:46.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:45 smithi089 ceph-mon[34495]: pgmap v183: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:46.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:45 smithi104 ceph-mon[34559]: pgmap v183: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:48.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:47 smithi003 ceph-mon[30137]: pgmap v184: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:48.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:47 smithi089 ceph-mon[34495]: pgmap v184: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:48.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:47 smithi104 ceph-mon[34559]: pgmap v184: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:49 smithi003 ceph-mon[30137]: pgmap v185: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:49 smithi089 ceph-mon[34495]: pgmap v185: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:50.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:49 smithi104 ceph-mon[34559]: pgmap v185: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:52.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:51 smithi003 ceph-mon[30137]: pgmap v186: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:52.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:51 smithi089 ceph-mon[34495]: pgmap v186: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:52.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:51 smithi104 ceph-mon[34559]: pgmap v186: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:54.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:53 smithi003 ceph-mon[30137]: pgmap v187: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:54.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:53 smithi089 ceph-mon[34495]: pgmap v187: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:53 smithi104 ceph-mon[34559]: pgmap v187: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T05:59:56.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:55 smithi003 ceph-mon[30137]: pgmap v188: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:55 smithi089 ceph-mon[34495]: pgmap v188: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:56.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:55 smithi104 ceph-mon[34559]: pgmap v188: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:58.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:57 smithi003 ceph-mon[30137]: pgmap v189: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:57 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:58.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:57 smithi089 ceph-mon[34495]: pgmap v189: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:58.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:57 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:58.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:57 smithi104 ceph-mon[34559]: pgmap v189: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T05:59:58.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:57 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:59:59.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:58 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:59.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:58 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:59.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:58 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:59:59.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:58 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:59.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:58 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:59.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:58 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T05:59:59.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:58 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:59:59.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:58 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:59:59.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:58 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:00:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 05:59:59 smithi003 ceph-mon[30137]: pgmap v190: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:00.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 05:59:59 smithi089 ceph-mon[34495]: pgmap v190: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:00.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 05:59:59 smithi104 ceph-mon[34559]: pgmap v190: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:01.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:00 smithi003 ceph-mon[30137]: overall HEALTH_OK 2024-05-08T06:00:01.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:00 smithi089 ceph-mon[34495]: overall HEALTH_OK 2024-05-08T06:00:01.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:00 smithi104 ceph-mon[34559]: overall HEALTH_OK 2024-05-08T06:00:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:01 smithi003 ceph-mon[30137]: pgmap v191: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:01 smithi089 ceph-mon[34495]: pgmap v191: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:02.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:01 smithi104 ceph-mon[34559]: pgmap v191: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:03 smithi003 ceph-mon[30137]: pgmap v192: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:00:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:03 smithi089 ceph-mon[34495]: pgmap v192: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:00:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:03 smithi104 ceph-mon[34559]: pgmap v192: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:00:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:05 smithi003 ceph-mon[30137]: pgmap v193: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:00:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:05 smithi089 ceph-mon[34495]: pgmap v193: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:00:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:05 smithi104 ceph-mon[34559]: pgmap v193: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:00:08.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:07 smithi003 ceph-mon[30137]: pgmap v194: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:00:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:07 smithi089 ceph-mon[34495]: pgmap v194: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:00:08.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:07 smithi104 ceph-mon[34559]: pgmap v194: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:00:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:09 smithi003 ceph-mon[30137]: pgmap v195: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:00:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:09 smithi003 ceph-mon[30137]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T06:00:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:09 smithi089 ceph-mon[34495]: pgmap v195: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:00:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:09 smithi089 ceph-mon[34495]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T06:00:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:09 smithi104 ceph-mon[34559]: pgmap v195: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:00:10.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:09 smithi104 ceph-mon[34559]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T06:00:12.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:11 smithi003 ceph-mon[30137]: pgmap v197: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:00:12.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:11 smithi089 ceph-mon[34495]: pgmap v197: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:00:12.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:11 smithi104 ceph-mon[34559]: pgmap v197: 129 pgs: 129 active+clean; 260 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:00:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:13 smithi003 ceph-mon[30137]: pgmap v198: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:00:14.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:13 smithi089 ceph-mon[34495]: pgmap v198: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:00:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:13 smithi104 ceph-mon[34559]: pgmap v198: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:00:15.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:15 smithi003 ceph-mon[30137]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T06:00:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:15 smithi003 ceph-mon[30137]: pgmap v200: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:00:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:15 smithi089 ceph-mon[34495]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T06:00:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:15 smithi089 ceph-mon[34495]: pgmap v200: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:00:15.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:15 smithi104 ceph-mon[34559]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T06:00:15.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:15 smithi104 ceph-mon[34559]: pgmap v200: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:00:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:17 smithi089 ceph-mon[34495]: pgmap v201: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:00:18.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:17 smithi104 ceph-mon[34559]: pgmap v201: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:00:18.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:17 smithi003 ceph-mon[30137]: pgmap v201: 129 pgs: 129 active+clean; 260 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:00:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:19 smithi089 ceph-mon[34495]: pgmap v202: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 210 B/s wr, 2 op/s 2024-05-08T06:00:20.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:19 smithi104 ceph-mon[34559]: pgmap v202: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 210 B/s wr, 2 op/s 2024-05-08T06:00:20.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:19 smithi003 ceph-mon[30137]: pgmap v202: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 210 B/s wr, 2 op/s 2024-05-08T06:00:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:21 smithi089 ceph-mon[34495]: pgmap v203: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:00:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:21 smithi104 ceph-mon[34559]: pgmap v203: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:00:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:21 smithi003 ceph-mon[30137]: pgmap v203: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:00:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:23 smithi089 ceph-mon[34495]: pgmap v204: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:00:24.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:23 smithi104 ceph-mon[34559]: pgmap v204: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:00:24.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:23 smithi003 ceph-mon[30137]: pgmap v204: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:00:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:25 smithi089 ceph-mon[34495]: pgmap v205: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:00:26.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:25 smithi104 ceph-mon[34559]: pgmap v205: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:00:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:25 smithi003 ceph-mon[30137]: pgmap v205: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:00:27.962 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:27 smithi003 ceph-mon[30137]: pgmap v206: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:00:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:27 smithi089 ceph-mon[34495]: pgmap v206: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:00:28.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:27 smithi104 ceph-mon[34559]: pgmap v206: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:00:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:29 smithi089 ceph-mon[34495]: pgmap v207: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:00:30.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:29 smithi104 ceph-mon[34559]: pgmap v207: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:00:30.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:29 smithi003 ceph-mon[30137]: pgmap v207: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:00:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:31 smithi089 ceph-mon[34495]: pgmap v208: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:31 smithi104 ceph-mon[34559]: pgmap v208: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:32.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:31 smithi003 ceph-mon[30137]: pgmap v208: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:33 smithi089 ceph-mon[34495]: pgmap v209: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:34.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:33 smithi104 ceph-mon[34559]: pgmap v209: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:34.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:33 smithi003 ceph-mon[30137]: pgmap v209: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:35 smithi089 ceph-mon[34495]: pgmap v210: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:36.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:35 smithi104 ceph-mon[34559]: pgmap v210: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:36.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:35 smithi003 ceph-mon[30137]: pgmap v210: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:37 smithi089 ceph-mon[34495]: pgmap v211: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:37 smithi104 ceph-mon[34559]: pgmap v211: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:38.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:37 smithi003 ceph-mon[30137]: pgmap v211: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:39 smithi089 ceph-mon[34495]: pgmap v212: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:39 smithi104 ceph-mon[34559]: pgmap v212: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:39 smithi003 ceph-mon[30137]: pgmap v212: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:41 smithi089 ceph-mon[34495]: pgmap v213: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:42.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:41 smithi104 ceph-mon[34559]: pgmap v213: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:42.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:41 smithi003 ceph-mon[30137]: pgmap v213: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:43 smithi089 ceph-mon[34495]: pgmap v214: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:43 smithi104 ceph-mon[34559]: pgmap v214: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:44.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:43 smithi003 ceph-mon[30137]: pgmap v214: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:45 smithi089 ceph-mon[34495]: pgmap v215: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:46.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:45 smithi104 ceph-mon[34559]: pgmap v215: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:46.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:45 smithi003 ceph-mon[30137]: pgmap v215: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:48.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:47 smithi089 ceph-mon[34495]: pgmap v216: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:48.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:47 smithi104 ceph-mon[34559]: pgmap v216: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:48.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:47 smithi003 ceph-mon[30137]: pgmap v216: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:50.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:49 smithi089 ceph-mon[34495]: pgmap v217: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:50.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:49 smithi104 ceph-mon[34559]: pgmap v217: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:49 smithi003 ceph-mon[30137]: pgmap v217: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:52.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:51 smithi089 ceph-mon[34495]: pgmap v218: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:52.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:51 smithi104 ceph-mon[34559]: pgmap v218: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:52.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:51 smithi003 ceph-mon[30137]: pgmap v218: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:53 smithi089 ceph-mon[34495]: pgmap v219: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:54.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:53 smithi104 ceph-mon[34559]: pgmap v219: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:54.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:53 smithi003 ceph-mon[30137]: pgmap v219: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:00:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:55 smithi089 ceph-mon[34495]: pgmap v220: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:56.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:55 smithi104 ceph-mon[34559]: pgmap v220: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:56.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:55 smithi003 ceph-mon[30137]: pgmap v220: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:57 smithi089 ceph-mon[34495]: pgmap v221: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:58.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:57 smithi104 ceph-mon[34559]: pgmap v221: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:58.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:57 smithi003 ceph-mon[30137]: pgmap v221: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:00:59.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:58 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:59.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:58 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:00:59.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:58 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:01:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:59 smithi089 ceph-mon[34495]: pgmap v222: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:59 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:59 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:00.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:00:59 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:01:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:59 smithi104 ceph-mon[34559]: pgmap v222: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:59 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:59 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:00.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:00:59 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:01:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:59 smithi003 ceph-mon[30137]: pgmap v222: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:59 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:01:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:59 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:01:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:00:59 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:01:02.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:01 smithi089 ceph-mon[34495]: pgmap v223: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:02.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:01 smithi104 ceph-mon[34559]: pgmap v223: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:01 smithi003 ceph-mon[30137]: pgmap v223: 129 pgs: 129 active+clean; 260 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:04.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:03 smithi089 ceph-mon[34495]: pgmap v224: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:01:04.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:03 smithi104 ceph-mon[34559]: pgmap v224: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:01:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:03 smithi003 ceph-mon[30137]: pgmap v224: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:01:06.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:05 smithi104 ceph-mon[34559]: pgmap v225: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:01:06.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:05 smithi003 ceph-mon[30137]: pgmap v225: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:01:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:05 smithi089 ceph-mon[34495]: pgmap v225: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:01:08.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:07 smithi104 ceph-mon[34559]: pgmap v226: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:01:08.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:07 smithi003 ceph-mon[30137]: pgmap v226: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:01:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:07 smithi089 ceph-mon[34495]: pgmap v226: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:01:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:09 smithi003 ceph-mon[30137]: pgmap v227: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:01:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:09 smithi003 ceph-mon[30137]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T06:01:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:09 smithi089 ceph-mon[34495]: pgmap v227: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:01:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:09 smithi089 ceph-mon[34495]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T06:01:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:09 smithi104 ceph-mon[34559]: pgmap v227: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:01:10.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:09 smithi104 ceph-mon[34559]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T06:01:12.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:11 smithi003 ceph-mon[30137]: pgmap v229: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:01:12.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:11 smithi089 ceph-mon[34495]: pgmap v229: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:01:12.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:11 smithi104 ceph-mon[34559]: pgmap v229: 129 pgs: 129 active+clean; 260 MiB data, 658 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:01:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:13 smithi003 ceph-mon[30137]: pgmap v230: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:01:14.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:13 smithi089 ceph-mon[34495]: pgmap v230: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:01:14.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:13 smithi104 ceph-mon[34559]: pgmap v230: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:01:15.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:15 smithi003 ceph-mon[30137]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T06:01:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:15 smithi003 ceph-mon[30137]: pgmap v232: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:01:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:15 smithi089 ceph-mon[34495]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T06:01:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:15 smithi089 ceph-mon[34495]: pgmap v232: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:01:15.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:15 smithi104 ceph-mon[34559]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T06:01:15.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:15 smithi104 ceph-mon[34559]: pgmap v232: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:01:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:17 smithi089 ceph-mon[34495]: pgmap v233: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:01:18.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:17 smithi104 ceph-mon[34559]: pgmap v233: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:01:18.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:17 smithi003 ceph-mon[30137]: pgmap v233: 129 pgs: 129 active+clean; 260 MiB data, 662 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:01:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:19 smithi089 ceph-mon[34495]: pgmap v234: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-05-08T06:01:20.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:19 smithi104 ceph-mon[34559]: pgmap v234: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-05-08T06:01:20.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:19 smithi003 ceph-mon[30137]: pgmap v234: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-05-08T06:01:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:21 smithi089 ceph-mon[34495]: pgmap v235: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:01:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:21 smithi104 ceph-mon[34559]: pgmap v235: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:01:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:21 smithi003 ceph-mon[30137]: pgmap v235: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:01:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:23 smithi089 ceph-mon[34495]: pgmap v236: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:01:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:23 smithi104 ceph-mon[34559]: pgmap v236: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:01:24.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:23 smithi003 ceph-mon[30137]: pgmap v236: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:01:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:25 smithi089 ceph-mon[34495]: pgmap v237: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:01:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:25 smithi104 ceph-mon[34559]: pgmap v237: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:01:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:25 smithi003 ceph-mon[30137]: pgmap v237: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:01:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:27 smithi089 ceph-mon[34495]: pgmap v238: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:01:28.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:27 smithi104 ceph-mon[34559]: pgmap v238: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:01:28.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:27 smithi003 ceph-mon[30137]: pgmap v238: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:01:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:29 smithi089 ceph-mon[34495]: pgmap v239: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:01:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:29 smithi104 ceph-mon[34559]: pgmap v239: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:01:30.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:29 smithi003 ceph-mon[30137]: pgmap v239: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:01:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:31 smithi089 ceph-mon[34495]: pgmap v240: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:31 smithi104 ceph-mon[34559]: pgmap v240: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:32.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:31 smithi003 ceph-mon[30137]: pgmap v240: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:33 smithi089 ceph-mon[34495]: pgmap v241: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:34.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:33 smithi104 ceph-mon[34559]: pgmap v241: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:34.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:33 smithi003 ceph-mon[30137]: pgmap v241: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:35.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:35 smithi003 ceph-mon[30137]: pgmap v242: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:35.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:35 smithi089 ceph-mon[34495]: pgmap v242: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:35 smithi104 ceph-mon[34559]: pgmap v242: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:37 smithi089 ceph-mon[34495]: pgmap v243: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:37 smithi104 ceph-mon[34559]: pgmap v243: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:38.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:37 smithi003 ceph-mon[30137]: pgmap v243: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:39 smithi089 ceph-mon[34495]: pgmap v244: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:40.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:39 smithi104 ceph-mon[34559]: pgmap v244: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:39 smithi003 ceph-mon[30137]: pgmap v244: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:42.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:41 smithi089 ceph-mon[34495]: pgmap v245: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:42.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:41 smithi104 ceph-mon[34559]: pgmap v245: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:42.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:41 smithi003 ceph-mon[30137]: pgmap v245: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:43 smithi089 ceph-mon[34495]: pgmap v246: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:43 smithi104 ceph-mon[34559]: pgmap v246: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:44.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:43 smithi003 ceph-mon[30137]: pgmap v246: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:45 smithi089 ceph-mon[34495]: pgmap v247: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:46.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:45 smithi104 ceph-mon[34559]: pgmap v247: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:46.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:45 smithi003 ceph-mon[30137]: pgmap v247: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:48.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:47 smithi089 ceph-mon[34495]: pgmap v248: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:48.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:47 smithi104 ceph-mon[34559]: pgmap v248: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:48.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:47 smithi003 ceph-mon[30137]: pgmap v248: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:50.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:49 smithi089 ceph-mon[34495]: pgmap v249: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:49 smithi104 ceph-mon[34559]: pgmap v249: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:49 smithi003 ceph-mon[30137]: pgmap v249: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:52.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:51 smithi089 ceph-mon[34495]: pgmap v250: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:52.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:51 smithi104 ceph-mon[34559]: pgmap v250: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:52.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:51 smithi003 ceph-mon[30137]: pgmap v250: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:54.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:53 smithi089 ceph-mon[34495]: pgmap v251: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:54.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:53 smithi104 ceph-mon[34559]: pgmap v251: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:54.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:53 smithi003 ceph-mon[30137]: pgmap v251: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:01:56.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:55 smithi089 ceph-mon[34495]: pgmap v252: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:56.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:55 smithi104 ceph-mon[34559]: pgmap v252: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:55 smithi003 ceph-mon[30137]: pgmap v252: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:58.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:57 smithi089 ceph-mon[34495]: pgmap v253: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:58.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:57 smithi104 ceph-mon[34559]: pgmap v253: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:01:58.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:57 smithi003 ceph-mon[30137]: pgmap v253: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:59 smithi089 ceph-mon[34495]: pgmap v254: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:00.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:01:59 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:00.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:59 smithi104 ceph-mon[34559]: pgmap v254: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:00.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:01:59 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:59 smithi003 ceph-mon[30137]: pgmap v254: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:00.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:01:59 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:02:01.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:01.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:02:01.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:01.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:01.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:02:01.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:02:01.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:02:01.163 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:02:02.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:01 smithi003 ceph-mon[30137]: pgmap v255: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:01 smithi089 ceph-mon[34495]: pgmap v255: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:02.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:01 smithi104 ceph-mon[34559]: pgmap v255: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:03 smithi003 ceph-mon[30137]: pgmap v256: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:02:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:03 smithi089 ceph-mon[34495]: pgmap v256: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:02:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:03 smithi104 ceph-mon[34559]: pgmap v256: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:02:06.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:05 smithi003 ceph-mon[30137]: pgmap v257: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:02:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:05 smithi089 ceph-mon[34495]: pgmap v257: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:02:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:05 smithi104 ceph-mon[34559]: pgmap v257: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:02:08.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:07 smithi003 ceph-mon[30137]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:02:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:07 smithi089 ceph-mon[34495]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:02:08.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:07 smithi104 ceph-mon[34559]: pgmap v258: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:02:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:09 smithi003 ceph-mon[30137]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:02:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:09 smithi003 ceph-mon[30137]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T06:02:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:09 smithi089 ceph-mon[34495]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:02:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:09 smithi089 ceph-mon[34495]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T06:02:10.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:09 smithi104 ceph-mon[34559]: pgmap v259: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:02:10.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:09 smithi104 ceph-mon[34559]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T06:02:12.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:11 smithi003 ceph-mon[30137]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:02:12.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:11 smithi089 ceph-mon[34495]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:02:12.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:11 smithi104 ceph-mon[34559]: pgmap v261: 129 pgs: 129 active+clean; 260 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:02:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:13 smithi003 ceph-mon[30137]: pgmap v262: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:02:14.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:13 smithi089 ceph-mon[34495]: pgmap v262: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:02:14.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:13 smithi104 ceph-mon[34559]: pgmap v262: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:02:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:15 smithi003 ceph-mon[30137]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T06:02:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:15 smithi003 ceph-mon[30137]: pgmap v264: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:02:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:15 smithi089 ceph-mon[34495]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T06:02:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:15 smithi089 ceph-mon[34495]: pgmap v264: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:02:15.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:15 smithi104 ceph-mon[34559]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T06:02:15.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:15 smithi104 ceph-mon[34559]: pgmap v264: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:02:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:17 smithi089 ceph-mon[34495]: pgmap v265: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:02:18.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:17 smithi104 ceph-mon[34559]: pgmap v265: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:02:18.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:17 smithi003 ceph-mon[30137]: pgmap v265: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:02:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:19 smithi089 ceph-mon[34495]: pgmap v266: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 209 B/s wr, 2 op/s 2024-05-08T06:02:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:19 smithi104 ceph-mon[34559]: pgmap v266: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 209 B/s wr, 2 op/s 2024-05-08T06:02:20.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:19 smithi003 ceph-mon[30137]: pgmap v266: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 209 B/s wr, 2 op/s 2024-05-08T06:02:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:21 smithi089 ceph-mon[34495]: pgmap v267: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:02:22.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:21 smithi104 ceph-mon[34559]: pgmap v267: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:02:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:21 smithi003 ceph-mon[30137]: pgmap v267: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:02:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:23 smithi089 ceph-mon[34495]: pgmap v268: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:02:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:23 smithi104 ceph-mon[34559]: pgmap v268: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:02:24.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:23 smithi003 ceph-mon[30137]: pgmap v268: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:02:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:25 smithi089 ceph-mon[34495]: pgmap v269: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:02:26.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:25 smithi104 ceph-mon[34559]: pgmap v269: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:02:26.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:25 smithi003 ceph-mon[30137]: pgmap v269: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:02:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:27 smithi089 ceph-mon[34495]: pgmap v270: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:02:28.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:27 smithi104 ceph-mon[34559]: pgmap v270: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:02:28.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:27 smithi003 ceph-mon[30137]: pgmap v270: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:02:30.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:29 smithi089 ceph-mon[34495]: pgmap v271: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:02:30.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:29 smithi104 ceph-mon[34559]: pgmap v271: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:02:30.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:29 smithi003 ceph-mon[30137]: pgmap v271: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:02:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:31 smithi089 ceph-mon[34495]: pgmap v272: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:32.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:31 smithi104 ceph-mon[34559]: pgmap v272: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:32.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:31 smithi003 ceph-mon[30137]: pgmap v272: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:34.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:33 smithi089 ceph-mon[34495]: pgmap v273: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:33 smithi104 ceph-mon[34559]: pgmap v273: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:34.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:33 smithi003 ceph-mon[30137]: pgmap v273: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:36.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:35 smithi089 ceph-mon[34495]: pgmap v274: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:36.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:35 smithi104 ceph-mon[34559]: pgmap v274: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:36.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:35 smithi003 ceph-mon[30137]: pgmap v274: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:37 smithi089 ceph-mon[34495]: pgmap v275: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:37 smithi104 ceph-mon[34559]: pgmap v275: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:38.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:37 smithi003 ceph-mon[30137]: pgmap v275: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:39 smithi089 ceph-mon[34495]: pgmap v276: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:39 smithi104 ceph-mon[34559]: pgmap v276: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:39 smithi003 ceph-mon[30137]: pgmap v276: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:41 smithi089 ceph-mon[34495]: pgmap v277: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:42.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:41 smithi104 ceph-mon[34559]: pgmap v277: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:42.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:41 smithi003 ceph-mon[30137]: pgmap v277: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:43 smithi089 ceph-mon[34495]: pgmap v278: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:44.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:43 smithi104 ceph-mon[34559]: pgmap v278: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:44.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:43 smithi003 ceph-mon[30137]: pgmap v278: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:45 smithi089 ceph-mon[34495]: pgmap v279: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:46.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:45 smithi104 ceph-mon[34559]: pgmap v279: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:46.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:45 smithi003 ceph-mon[30137]: pgmap v279: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:48.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:47 smithi104 ceph-mon[34559]: pgmap v280: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:48.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:47 smithi003 ceph-mon[30137]: pgmap v280: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:48.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:47 smithi089 ceph-mon[34495]: pgmap v280: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:49 smithi104 ceph-mon[34559]: pgmap v281: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:50.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:49 smithi003 ceph-mon[30137]: pgmap v281: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:49 smithi089 ceph-mon[34495]: pgmap v281: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:52.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:51 smithi104 ceph-mon[34559]: pgmap v282: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:52.065 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:51 smithi089 ceph-mon[34495]: pgmap v282: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:52.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:51 smithi003 ceph-mon[30137]: pgmap v282: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:54.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:53 smithi003 ceph-mon[30137]: pgmap v283: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:54.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:53 smithi089 ceph-mon[34495]: pgmap v283: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:53 smithi104 ceph-mon[34559]: pgmap v283: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:02:56.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:55 smithi003 ceph-mon[30137]: pgmap v284: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:55 smithi089 ceph-mon[34495]: pgmap v284: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:56.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:55 smithi104 ceph-mon[34559]: pgmap v284: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:58.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:57 smithi003 ceph-mon[30137]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:58.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:57 smithi089 ceph-mon[34495]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:02:58.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:57 smithi104 ceph-mon[34559]: pgmap v285: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:00.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:02:59 smithi003 ceph-mon[30137]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:00.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:02:59 smithi089 ceph-mon[34495]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:00.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:02:59 smithi104 ceph-mon[34559]: pgmap v286: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:01.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:01.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:01.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:01.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:00 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:03:01.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:01.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:01.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:01.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:00 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:03:01.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:03:01.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:03:01.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:03:01.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:00 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:03:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:01 smithi003 ceph-mon[30137]: pgmap v287: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:01 smithi089 ceph-mon[34495]: pgmap v287: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:02.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:01 smithi104 ceph-mon[34559]: pgmap v287: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:04.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:03 smithi003 ceph-mon[30137]: pgmap v288: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:03:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:03 smithi089 ceph-mon[34495]: pgmap v288: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:03:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:03 smithi104 ceph-mon[34559]: pgmap v288: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:03:06.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:05 smithi003 ceph-mon[30137]: pgmap v289: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:03:06.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:05 smithi089 ceph-mon[34495]: pgmap v289: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:03:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:05 smithi104 ceph-mon[34559]: pgmap v289: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:03:08.162 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:07 smithi003 ceph-mon[30137]: pgmap v290: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:03:08.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:07 smithi089 ceph-mon[34495]: pgmap v290: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:03:08.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:07 smithi104 ceph-mon[34559]: pgmap v290: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:03:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:09 smithi003 ceph-mon[30137]: pgmap v291: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:03:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:09 smithi003 ceph-mon[30137]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T06:03:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:09 smithi089 ceph-mon[34495]: pgmap v291: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:03:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:09 smithi089 ceph-mon[34495]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T06:03:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:09 smithi104 ceph-mon[34559]: pgmap v291: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:03:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:09 smithi104 ceph-mon[34559]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T06:03:12.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:11 smithi003 ceph-mon[30137]: pgmap v293: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:03:12.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:11 smithi089 ceph-mon[34495]: pgmap v293: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:03:12.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:11 smithi104 ceph-mon[34559]: pgmap v293: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:03:14.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:13 smithi003 ceph-mon[30137]: pgmap v294: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:03:14.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:13 smithi089 ceph-mon[34495]: pgmap v294: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:03:14.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:13 smithi104 ceph-mon[34559]: pgmap v294: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:03:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:15 smithi003 ceph-mon[30137]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T06:03:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:15 smithi003 ceph-mon[30137]: pgmap v296: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:03:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:15 smithi089 ceph-mon[34495]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T06:03:15.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:15 smithi089 ceph-mon[34495]: pgmap v296: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:03:15.770 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:15 smithi104 ceph-mon[34559]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T06:03:15.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:15 smithi104 ceph-mon[34559]: pgmap v296: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:03:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:17 smithi089 ceph-mon[34495]: pgmap v297: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:03:18.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:17 smithi104 ceph-mon[34559]: pgmap v297: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:03:18.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:17 smithi003 ceph-mon[30137]: pgmap v297: 129 pgs: 129 active+clean; 2.5 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:03:20.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:19 smithi089 ceph-mon[34495]: pgmap v298: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-05-08T06:03:20.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:19 smithi104 ceph-mon[34559]: pgmap v298: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-05-08T06:03:20.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:19 smithi003 ceph-mon[30137]: pgmap v298: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 207 B/s wr, 2 op/s 2024-05-08T06:03:22.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:21 smithi089 ceph-mon[34495]: pgmap v299: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:03:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:21 smithi104 ceph-mon[34559]: pgmap v299: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:03:22.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:21 smithi003 ceph-mon[30137]: pgmap v299: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:03:24.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:23 smithi089 ceph-mon[34495]: pgmap v300: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:03:24.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:23 smithi104 ceph-mon[34559]: pgmap v300: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:03:24.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:23 smithi003 ceph-mon[30137]: pgmap v300: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:03:26.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:25 smithi089 ceph-mon[34495]: pgmap v301: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:03:26.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:25 smithi104 ceph-mon[34559]: pgmap v301: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:03:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:25 smithi003 ceph-mon[30137]: pgmap v301: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:03:28.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:27 smithi089 ceph-mon[34495]: pgmap v302: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:03:28.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:27 smithi104 ceph-mon[34559]: pgmap v302: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:03:28.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:27 smithi003 ceph-mon[30137]: pgmap v302: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:03:30.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:29 smithi089 ceph-mon[34495]: pgmap v303: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:03:30.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:29 smithi104 ceph-mon[34559]: pgmap v303: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:03:30.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:29 smithi003 ceph-mon[30137]: pgmap v303: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:03:32.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:31 smithi089 ceph-mon[34495]: pgmap v304: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:32.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:31 smithi104 ceph-mon[34559]: pgmap v304: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:32.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:31 smithi003 ceph-mon[30137]: pgmap v304: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:34.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:33 smithi104 ceph-mon[34559]: pgmap v305: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:34.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:33 smithi003 ceph-mon[30137]: pgmap v305: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:34.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:33 smithi089 ceph-mon[34495]: pgmap v305: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:35.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:35 smithi003 ceph-mon[30137]: pgmap v306: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:35.762 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:35 smithi089 ceph-mon[34495]: pgmap v306: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:35.771 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:35 smithi104 ceph-mon[34559]: pgmap v306: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:38.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:37 smithi089 ceph-mon[34495]: pgmap v307: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:38.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:37 smithi104 ceph-mon[34559]: pgmap v307: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:38.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:37 smithi003 ceph-mon[30137]: pgmap v307: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:40.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:39 smithi089 ceph-mon[34495]: pgmap v308: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:40.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:39 smithi104 ceph-mon[34559]: pgmap v308: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:39 smithi003 ceph-mon[30137]: pgmap v308: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:42.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:41 smithi089 ceph-mon[34495]: pgmap v309: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:42.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:41 smithi104 ceph-mon[34559]: pgmap v309: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:42.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:41 smithi003 ceph-mon[30137]: pgmap v309: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:44.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:43 smithi089 ceph-mon[34495]: pgmap v310: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:44.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:43 smithi104 ceph-mon[34559]: pgmap v310: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:44.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:43 smithi003 ceph-mon[30137]: pgmap v310: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:46.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:45 smithi089 ceph-mon[34495]: pgmap v311: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:46.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:45 smithi104 ceph-mon[34559]: pgmap v311: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:46.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:45 smithi003 ceph-mon[30137]: pgmap v311: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:48.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:47 smithi089 ceph-mon[34495]: pgmap v312: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:48.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:47 smithi104 ceph-mon[34559]: pgmap v312: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:48.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:47 smithi003 ceph-mon[30137]: pgmap v312: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:48.319 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T06:03:48.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 -- 172.21.15.3:0/771095534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e08103370 msgr2=0x7f9e0810f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.441 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 --2- 172.21.15.3:0/771095534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e08103370 0x7f9e0810f880 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009930 tx=0x7f9e0002a420 comp rx=0 tx=0).stop 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 -- 172.21.15.3:0/771095534 shutdown_connections 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 --2- 172.21.15.3:0/771095534 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e08103370 0x7f9e0810f880 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 --2- 172.21.15.3:0/771095534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e081029b0 0x7f9e08102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 --2- 172.21.15.3:0/771095534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e081089c0 0x7f9e08108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 -- 172.21.15.3:0/771095534 >> 172.21.15.3:0/771095534 conn(0x7f9e080fe4a0 msgr2=0x7f9e081008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 -- 172.21.15.3:0/771095534 shutdown_connections 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.438+0000 7f9e0d450640 1 -- 172.21.15.3:0/771095534 wait complete. 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 Processor -- start 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 -- start start 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 0x7f9e081991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e08103370 0x7f9e081996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e081089c0 0x7f9e0819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.442 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08112c80 con 0x7f9e081029b0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08112b00 con 0x7f9e081089c0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e0d450640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e08112e00 con 0x7f9e08103370 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e06ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 0x7f9e081991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e06ffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 0x7f9e081991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40168/0 (socket says 172.21.15.3:40168) 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e06ffd640 1 -- 172.21.15.3:0/1836459248 learned_addr learned my addr 172.21.15.3:0/1836459248 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e077fe640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e081089c0 0x7f9e0819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.439+0000 7f9e067fc640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e08103370 0x7f9e081996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e06ffd640 1 -- 172.21.15.3:0/1836459248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e08103370 msgr2=0x7f9e081996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e06ffd640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e08103370 0x7f9e081996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e06ffd640 1 -- 172.21.15.3:0/1836459248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e081089c0 msgr2=0x7f9e0819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e06ffd640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e081089c0 0x7f9e0819da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e06ffd640 1 -- 172.21.15.3:0/1836459248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e0819e190 con 0x7f9e081029b0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e06ffd640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 0x7f9e081991a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9df800ea80 tx=0x7f9df800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:48.443 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df800ce40 con 0x7f9e081029b0 2024-05-08T06:03:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e0819e480 con 0x7f9e081029b0 2024-05-08T06:03:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9df8005020 con 0x7f9e081029b0 2024-05-08T06:03:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9df80116d0 con 0x7f9e081029b0 2024-05-08T06:03:48.444 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.440+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e081a5d10 con 0x7f9e081029b0 2024-05-08T06:03:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.441+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e08050f90 con 0x7f9e081029b0 2024-05-08T06:03:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.443+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f9df80054d0 con 0x7f9e081029b0 2024-05-08T06:03:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.443+0000 7f9dfffff640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f9df40b2950 0x7f9df40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.443+0000 7f9e067fc640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f9df40b2950 0x7f9df40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.443+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f9df80e2bb0 con 0x7f9e081029b0 2024-05-08T06:03:48.445 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.444+0000 7f9e067fc640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f9df40b2950 0x7f9df40b4e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9e08100b90 tx=0x7f9df000b390 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:48.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.448+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9df8016030 con 0x7f9e081029b0 2024-05-08T06:03:48.664 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.663+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9e08062df0 con 0x7f9df40b2950 2024-05-08T06:03:48.672 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.671+0000 7f9dfffff640 1 -- 172.21.15.3:0/1836459248 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f9e08062df0 con 0x7f9df40b2950 2024-05-08T06:03:48.672 INFO:teuthology.orchestra.run.smithi003.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T05:53:47", "first": "2024-05-08T05:54:00", "last": "2024-05-08T06:03:00", "last_pruned": "2024-05-08T06:03:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-08T06:03:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f9df40b2950 msgr2=0x7f9df40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f9df40b2950 0x7f9df40b4e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9e08100b90 tx=0x7f9df000b390 comp rx=0 tx=0).stop 2024-05-08T06:03:48.674 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 msgr2=0x7f9e081991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 0x7f9e081991a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9df800ea80 tx=0x7f9df800ef40 comp rx=0 tx=0).stop 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 shutdown_connections 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f9df40b2950 0x7f9df40b4e10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9e081089c0 0x7f9e0819da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e08103370 0x7f9e081996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 --2- 172.21.15.3:0/1836459248 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f9e081029b0 0x7f9e081991a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 >> 172.21.15.3:0/1836459248 conn(0x7f9e080fe4a0 msgr2=0x7f9e080fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 shutdown_connections 2024-05-08T06:03:48.675 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.673+0000 7f9e0d450640 1 -- 172.21.15.3:0/1836459248 wait complete. 2024-05-08T06:03:48.685 DEBUG:teuthology.orchestra.run.smithi003:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T06:03:48.796 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 -- 172.21.15.3:0/3731331119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e8103370 msgr2=0x7f04e810f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/3731331119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e8103370 0x7f04e810f880 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009cd0 tx=0x7f04e002a8e0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 -- 172.21.15.3:0/3731331119 shutdown_connections 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/3731331119 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e8103370 0x7f04e810f880 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/3731331119 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e81029b0 0x7f04e8102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/3731331119 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e81089c0 0x7f04e8108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 -- 172.21.15.3:0/3731331119 >> 172.21.15.3:0/3731331119 conn(0x7f04e80fe4a0 msgr2=0x7f04e81008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 -- 172.21.15.3:0/3731331119 shutdown_connections 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.793+0000 7f04ef7f0640 1 -- 172.21.15.3:0/3731331119 wait complete. 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 Processor -- start 2024-05-08T06:03:48.797 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 -- start start 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 0x7f04e81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e8103370 0x7f04e8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e81089c0 0x7f04e819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8112c80 con 0x7f04e81029b0 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8112b00 con 0x7f04e81089c0 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ef7f0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04e8112e00 con 0x7f04e8103370 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 0x7f04e81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 0x7f04e81991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40184/0 (socket says 172.21.15.3:40184) 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 -- 172.21.15.3:0/4032008236 learned_addr learned my addr 172.21.15.3:0/4032008236 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ecd8b640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e8103370 0x7f04e8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04edd8d640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e81089c0 0x7f04e819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.798 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 -- 172.21.15.3:0/4032008236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e8103370 msgr2=0x7f04e8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e8103370 0x7f04e8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 -- 172.21.15.3:0/4032008236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e81089c0 msgr2=0x7f04e819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e81089c0 0x7f04e819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.794+0000 7f04ed58c640 1 -- 172.21.15.3:0/4032008236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e819e180 con 0x7f04e81029b0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.795+0000 7f04ed58c640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 0x7f04e81991e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f04e400d950 tx=0x7f04e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.795+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e4017070 con 0x7f04e81029b0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.795+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04e4009c30 con 0x7f04e81029b0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.795+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e819e470 con 0x7f04e81029b0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.795+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e400f620 con 0x7f04e81029b0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.795+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81a5d00 con 0x7f04e81029b0 2024-05-08T06:03:48.799 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.796+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8050f90 con 0x7f04e81029b0 2024-05-08T06:03:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.798+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f04e4005030 con 0x7f04e81029b0 2024-05-08T06:03:48.800 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.798+0000 7f04de7fc640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f04d40b29d0 0x7f04d40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.798+0000 7f04ecd8b640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f04d40b29d0 0x7f04d40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.798+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f04e40e24e0 con 0x7f04e81029b0 2024-05-08T06:03:48.801 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.799+0000 7f04ecd8b640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f04d40b29d0 0x7f04d40b4e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d8005850 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:48.803 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:48.802+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04e40a9c10 con 0x7f04e81029b0 2024-05-08T06:03:49.020 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.019+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 --> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f04e8062df0 con 0x7f04d40b29d0 2024-05-08T06:03:49.027 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.026+0000 7f04de7fc640 1 -- 172.21.15.3:0/4032008236 <== mgr.15129 v2:172.21.15.3:6800/3346608619 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f04e8062df0 con 0x7f04d40b29d0 2024-05-08T06:03:49.027 INFO:teuthology.orchestra.run.smithi003.stdout:/ 1m 6m3h 2024-05-08T06:03:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f04d40b29d0 msgr2=0x7f04d40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.029 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f04d40b29d0 0x7f04d40b4e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d8005850 comp rx=0 tx=0).stop 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 msgr2=0x7f04e81991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 0x7f04e81991e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f04e400d950 tx=0x7f04e400de10 comp rx=0 tx=0).stop 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 shutdown_connections 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f04d40b29d0 0x7f04d40b4e90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f04e81089c0 0x7f04e819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04e8103370 0x7f04e8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 --2- 172.21.15.3:0/4032008236 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f04e81029b0 0x7f04e81991e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 >> 172.21.15.3:0/4032008236 conn(0x7f04e80fe4a0 msgr2=0x7f04e80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 shutdown_connections 2024-05-08T06:03:49.030 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.028+0000 7f04ef7f0640 1 -- 172.21.15.3:0/4032008236 wait complete. 2024-05-08T06:03:49.041 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T06:03:49.053 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T06:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 -- 172.21.15.3:0/1617884688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb01089a0 msgr2=0x7f8fb0108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/1617884688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb01089a0 0x7f8fb0108d80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009930 tx=0x7f8fa402a420 comp rx=0 tx=0).stop 2024-05-08T06:03:49.160 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 -- 172.21.15.3:0/1617884688 shutdown_connections 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/1617884688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fb0103350 0x7f8fb010f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/1617884688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb0102990 0x7f8fb0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/1617884688 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb01089a0 0x7f8fb0108d80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 -- 172.21.15.3:0/1617884688 >> 172.21.15.3:0/1617884688 conn(0x7f8fb0079e40 msgr2=0x7f8fb007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 -- 172.21.15.3:0/1617884688 shutdown_connections 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.157+0000 7f8fb6de4640 1 -- 172.21.15.3:0/1617884688 wait complete. 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 Processor -- start 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 -- start start 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fb0102990 0x7f8fb0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 0x7f8fb0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb01089a0 0x7f8fb019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0112c60 con 0x7f8fb0103350 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8faffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 0x7f8fb0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0112ae0 con 0x7f8fb01089a0 2024-05-08T06:03:49.161 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb5381640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb01089a0 0x7f8fb019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb6de4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb0112de0 con 0x7f8fb0102990 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8faffff640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 0x7f8fb0199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40202/0 (socket says 172.21.15.3:40202) 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8fb4b80640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fb0102990 0x7f8fb0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.158+0000 7f8faffff640 1 -- 172.21.15.3:0/2994875512 learned_addr learned my addr 172.21.15.3:0/2994875512 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8faffff640 1 -- 172.21.15.3:0/2994875512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fb0102990 msgr2=0x7f8fb0199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8faffff640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fb0102990 0x7f8fb0199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8faffff640 1 -- 172.21.15.3:0/2994875512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb01089a0 msgr2=0x7f8fb019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8faffff640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb01089a0 0x7f8fb019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8faffff640 1 -- 172.21.15.3:0/2994875512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb019e110 con 0x7f8fb0103350 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8faffff640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 0x7f8fb0199660 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00c920 tx=0x7f8f9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c007c70 con 0x7f8fb0103350 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f9c007e10 con 0x7f8fb0103350 2024-05-08T06:03:49.162 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb019e400 con 0x7f8fb0103350 2024-05-08T06:03:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c018820 con 0x7f8fb0103350 2024-05-08T06:03:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.159+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb01a5c90 con 0x7f8fb0103350 2024-05-08T06:03:49.163 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.160+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb0069cb0 con 0x7f8fb0103350 2024-05-08T06:03:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.161+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f8f9c00f450 con 0x7f8fb0103350 2024-05-08T06:03:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.162+0000 7f8fadffb640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8fa00b2960 0x7f8fa00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.162+0000 7f8fb4b80640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8fa00b2960 0x7f8fa00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.162+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f8f9c0e2b00 con 0x7f8fb0103350 2024-05-08T06:03:49.164 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.164+0000 7f8fb4b80640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8fa00b2960 0x7f8fa00b4e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009900 tx=0x7f8fa4000bc0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:49.167 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.167+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f9c014030 con 0x7f8fb0103350 2024-05-08T06:03:49.359 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.359+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fb010af00 con 0x7f8fb0103350 2024-05-08T06:03:49.361 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.361+0000 7f8fadffb640 1 -- 172.21.15.3:0/2994875512 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+25218 (secure 0 0 0) 0x7f8f9c0aa0b0 con 0x7f8fb0103350 2024-05-08T06:03:49.362 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T06:03:49.362 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":99,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T06:03:14.380944+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":98,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2793699832":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/864931570":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/3161157856":"2024-05-09T05:53:46.504286+0000","172.21.15.3:6800/2793699832":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/1128678241":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/3921401880":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:03:49.363 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8fa00b2960 msgr2=0x7f8fa00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8fa00b2960 0x7f8fa00b4e20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009900 tx=0x7f8fa4000bc0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 msgr2=0x7f8fb0199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 0x7f8fb0199660 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00c920 tx=0x7f8f9c00cde0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 shutdown_connections 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f8fa00b2960 0x7f8fa00b4e20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8fb01089a0 0x7f8fb019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f8fb0103350 0x7f8fb0199660 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 --2- 172.21.15.3:0/2994875512 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fb0102990 0x7f8fb0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 >> 172.21.15.3:0/2994875512 conn(0x7f8fb0079e40 msgr2=0x7f8fb01003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 shutdown_connections 2024-05-08T06:03:49.364 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.363+0000 7f8fb6de4640 1 -- 172.21.15.3:0/2994875512 wait complete. 2024-05-08T06:03:49.375 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:45:01.214856+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T05:49:54.940678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T05:49:58.453825+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T05:50:10.686434+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 98, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T06:03:49.376 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 -- 172.21.15.3:0/1099160894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 msgr2=0x7fa25c10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 --2- 172.21.15.3:0/1099160894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c10e860 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa254009ca0 tx=0x7fa25402a940 comp rx=0 tx=0).stop 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 -- 172.21.15.3:0/1099160894 shutdown_connections 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 --2- 172.21.15.3:0/1099160894 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c10e860 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 --2- 172.21.15.3:0/1099160894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25c103390 0x7fa25c1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 --2- 172.21.15.3:0/1099160894 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 0x7fa25c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 -- 172.21.15.3:0/1099160894 >> 172.21.15.3:0/1099160894 conn(0x7fa25c0789d0 msgr2=0x7fa25c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 -- 172.21.15.3:0/1099160894 shutdown_connections 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.483+0000 7fa262646640 1 -- 172.21.15.3:0/1099160894 wait complete. 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 Processor -- start 2024-05-08T06:03:49.484 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 -- start start 2024-05-08T06:03:49.486 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 0x7fa25c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25c103390 0x7fa25c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c1a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c111ca0 con 0x7fa25c103d30 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c111b20 con 0x7fa25c103390 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa262646640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa25c111e20 con 0x7fa25c101c50 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa260be3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c1a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa260be3640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c1a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40210/0 (socket says 172.21.15.3:40210) 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa260be3640 1 -- 172.21.15.3:0/3505083245 learned_addr learned my addr 172.21.15.3:0/3505083245 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.484+0000 7fa25b7fe640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25c103390 0x7fa25c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa25bfff640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 0x7fa25c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa260be3640 1 -- 172.21.15.3:0/3505083245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 msgr2=0x7fa25c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa260be3640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 0x7fa25c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.487 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa260be3640 1 -- 172.21.15.3:0/3505083245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25c103390 msgr2=0x7fa25c19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa260be3640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25c103390 0x7fa25c19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa260be3640 1 -- 172.21.15.3:0/3505083245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25c1a56e0 con 0x7fa25c103d30 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa25bfff640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 0x7fa25c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa260be3640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c1a51a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa254004af0 tx=0x7fa254003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254034040 con 0x7fa25c103d30 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa254031e80 con 0x7fa25c103d30 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25c1a58b0 con 0x7fa25c103d30 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2540046c0 con 0x7fa25c103d30 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.485+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa25c1a5d70 con 0x7fa25c103d30 2024-05-08T06:03:49.488 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.486+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25c069cb0 con 0x7fa25c103d30 2024-05-08T06:03:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.488+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fa254037020 con 0x7fa25c103d30 2024-05-08T06:03:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.489+0000 7fa2597fa640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa2440b29e0 0x7fa2440b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.489+0000 7fa25bfff640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa2440b29e0 0x7fa2440b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.489+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fa254103880 con 0x7fa25c103d30 2024-05-08T06:03:49.489 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.489+0000 7fa25bfff640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa2440b29e0 0x7fa2440b4ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0058c0 tx=0x7fa24c005d30 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:49.492 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.492+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2540cafa0 con 0x7fa25c103d30 2024-05-08T06:03:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.684+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa25c109eb0 con 0x7fa25c103d30 2024-05-08T06:03:49.684 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.684+0000 7fa2597fa640 1 -- 172.21.15.3:0/3505083245 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7fa2540ca8a0 con 0x7fa25c103d30 2024-05-08T06:03:49.684 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 1 2024-05-08T06:03:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa2440b29e0 msgr2=0x7fa2440b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.686 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa2440b29e0 0x7fa2440b4ea0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0058c0 tx=0x7fa24c005d30 comp rx=0 tx=0).stop 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 msgr2=0x7fa25c1a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c1a51a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa254004af0 tx=0x7fa254003ba0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 shutdown_connections 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fa2440b29e0 0x7fa2440b4ea0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fa25c103d30 0x7fa25c1a51a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa25c103390 0x7fa25c19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 --2- 172.21.15.3:0/3505083245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa25c101c50 0x7fa25c19a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 >> 172.21.15.3:0/3505083245 conn(0x7fa25c0789d0 msgr2=0x7fa25c1067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 shutdown_connections 2024-05-08T06:03:49.687 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.686+0000 7fa262646640 1 -- 172.21.15.3:0/3505083245 wait complete. 2024-05-08T06:03:49.697 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T06:03:49.807 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 -- 172.21.15.3:0/80936898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c1045d0 msgr2=0x7f4a4c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 --2- 172.21.15.3:0/80936898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c1045d0 0x7f4a4c10ca20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009900 tx=0x7f4a4002a410 comp rx=0 tx=0).stop 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 -- 172.21.15.3:0/80936898 shutdown_connections 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 --2- 172.21.15.3:0/80936898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a4c10d130 0x7f4a4c10f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 --2- 172.21.15.3:0/80936898 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c1045d0 0x7f4a4c10ca20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 --2- 172.21.15.3:0/80936898 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a4c103c20 0x7f4a4c104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 -- 172.21.15.3:0/80936898 >> 172.21.15.3:0/80936898 conn(0x7f4a4c0789d0 msgr2=0x7f4a4c078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 -- 172.21.15.3:0/80936898 shutdown_connections 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.805+0000 7f4a524a2640 1 -- 172.21.15.3:0/80936898 wait complete. 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 Processor -- start 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 -- start start 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a4c103c20 0x7f4a4c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.808 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 0x7f4a4c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a4c19dca0 0x7f4a4c19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c112e20 con 0x7f4a4c10d130 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c112ca0 con 0x7f4a4c19dca0 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.806+0000 7f4a524a2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c112fa0 con 0x7f4a4c103c20 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 0x7f4a4c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 0x7f4a4c199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40222/0 (socket says 172.21.15.3:40222) 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 -- 172.21.15.3:0/2911058187 learned_addr learned my addr 172.21.15.3:0/2911058187 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a50a3f640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a4c19dca0 0x7f4a4c19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4bfff640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a4c103c20 0x7f4a4c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 -- 172.21.15.3:0/2911058187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a4c103c20 msgr2=0x7f4a4c1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a4c103c20 0x7f4a4c1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 -- 172.21.15.3:0/2911058187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a4c19dca0 msgr2=0x7f4a4c19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:49.809 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a4c19dca0 0x7f4a4c19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 -- 172.21.15.3:0/2911058187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c19e790 con 0x7f4a4c10d130 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a4b7fe640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 0x7f4a4c199920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a40031440 tx=0x7f4a40003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4003f070 con 0x7f4a4c10d130 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a40031d60 con 0x7f4a4c10d130 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c19ea20 con 0x7f4a4c10d130 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a40035030 con 0x7f4a4c10d130 2024-05-08T06:03:49.810 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.807+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1a6190 con 0x7f4a4c10d130 2024-05-08T06:03:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.808+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c069cb0 con 0x7f4a4c10d130 2024-05-08T06:03:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.810+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f4a40042070 con 0x7f4a4c10d130 2024-05-08T06:03:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.810+0000 7f4a497fa640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a340b2960 0x7f4a340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.811+0000 7f4a4bfff640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a340b2960 0x7f4a340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.811+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f4a40103730 con 0x7f4a4c10d130 2024-05-08T06:03:49.811 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.811+0000 7f4a4bfff640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a340b2960 0x7f4a340b4e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0058c0 tx=0x7f4a3c00a4c0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:49.813 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:49.814+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4a400cace0 con 0x7f4a4c10d130 2024-05-08T06:03:50.003 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.003+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4a4c19a730 con 0x7f4a4c10d130 2024-05-08T06:03:50.004 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.004+0000 7f4a497fa640 1 -- 172.21.15.3:0/2911058187 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) ==== 104+0+11 (secure 0 0 0) 0x7f4a400ca5e0 con 0x7f4a4c10d130 2024-05-08T06:03:50.004 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a340b2960 msgr2=0x7f4a340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a340b2960 0x7f4a340b4e20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0058c0 tx=0x7f4a3c00a4c0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 msgr2=0x7f4a4c199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 0x7f4a4c199920 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a40031440 tx=0x7f4a40003d50 comp rx=0 tx=0).stop 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 shutdown_connections 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f4a340b2960 0x7f4a340b4e20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.006 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a4c19dca0 0x7f4a4c19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f4a4c10d130 0x7f4a4c199920 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 --2- 172.21.15.3:0/2911058187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4a4c103c20 0x7f4a4c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 >> 172.21.15.3:0/2911058187 conn(0x7f4a4c0789d0 msgr2=0x7f4a4c0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 shutdown_connections 2024-05-08T06:03:50.007 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.006+0000 7f4a524a2640 1 -- 172.21.15.3:0/2911058187 wait complete. 2024-05-08T06:03:50.015 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:49 smithi003 ceph-mon[30137]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:50.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:49 smithi003 ceph-mon[30137]: pgmap v313: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:50.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:49 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2994875512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:03:50.016 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:49 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3505083245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:03:50.017 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T06:03:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:49 smithi104 ceph-mon[34559]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:49 smithi104 ceph-mon[34559]: pgmap v313: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:49 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2994875512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:03:50.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:49 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3505083245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:03:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 -- 172.21.15.3:0/2135029158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c108980 msgr2=0x7fca5c108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.126 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2135029158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c108980 0x7fca5c108d60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009930 tx=0x7fca4c02a440 comp rx=0 tx=0).stop 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 -- 172.21.15.3:0/2135029158 shutdown_connections 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2135029158 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c103330 0x7fca5c10f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2135029158 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5c102970 0x7fca5c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2135029158 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c108980 0x7fca5c108d60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 -- 172.21.15.3:0/2135029158 >> 172.21.15.3:0/2135029158 conn(0x7fca5c0fe460 msgr2=0x7fca5c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 -- 172.21.15.3:0/2135029158 shutdown_connections 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.124+0000 7fca61ef8640 1 -- 172.21.15.3:0/2135029158 wait complete. 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 Processor -- start 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 -- start start 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 0x7fca5c074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5c103330 0x7fca5c0751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.127 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c108980 0x7fca5c075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c112c10 con 0x7fca5c102970 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c112a90 con 0x7fca5c108980 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca61ef8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c112d90 con 0x7fca5c103330 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca5b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 0x7fca5c074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca5b7fe640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 0x7fca5c074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40238/0 (socket says 172.21.15.3:40238) 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca5b7fe640 1 -- 172.21.15.3:0/2728007068 learned_addr learned my addr 172.21.15.3:0/2728007068 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.125+0000 7fca5affd640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5c103330 0x7fca5c0751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5bfff640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c108980 0x7fca5c075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5b7fe640 1 -- 172.21.15.3:0/2728007068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5c103330 msgr2=0x7fca5c0751f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5b7fe640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5c103330 0x7fca5c0751f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5b7fe640 1 -- 172.21.15.3:0/2728007068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c108980 msgr2=0x7fca5c075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.128 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5b7fe640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c108980 0x7fca5c075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5b7fe640 1 -- 172.21.15.3:0/2728007068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c1a7630 con 0x7fca5c102970 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5bfff640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c108980 0x7fca5c075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca5b7fe640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 0x7fca5c074cb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca4c005940 tx=0x7fca4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c03f030 con 0x7fca5c102970 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c003aa0 con 0x7fca5c102970 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4c035070 con 0x7fca5c102970 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c1a78c0 con 0x7fca5c102970 2024-05-08T06:03:50.129 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.126+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c1a7db0 con 0x7fca5c102970 2024-05-08T06:03:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.127+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c069cb0 con 0x7fca5c102970 2024-05-08T06:03:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.128+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fca4c042030 con 0x7fca5c102970 2024-05-08T06:03:50.130 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.129+0000 7fca58ff9640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fca440b2960 0x7fca440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.129+0000 7fca5affd640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fca440b2960 0x7fca440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.129+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7fca4c102ff0 con 0x7fca5c102970 2024-05-08T06:03:50.131 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.130+0000 7fca5affd640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fca440b2960 0x7fca440b4e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fca5c076740 tx=0x7fca5000a2d0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:50.133 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.134+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca4c0ca5e0 con 0x7fca5c102970 2024-05-08T06:03:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:49 smithi089 ceph-mon[34495]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:49 smithi089 ceph-mon[34495]: pgmap v313: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-08T06:03:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:49 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2994875512' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:03:50.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:49 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3505083245' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T06:03:50.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.324+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fca5c02db50 con 0x7fca5c102970 2024-05-08T06:03:50.324 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.324+0000 7fca58ff9640 1 -- 172.21.15.3:0/2728007068 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) ==== 100+0+11 (secure 0 0 0) 0x7fca4c0c9ee0 con 0x7fca5c102970 2024-05-08T06:03:50.325 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 64 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.326+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fca440b2960 msgr2=0x7fca440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.326+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fca440b2960 0x7fca440b4e20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fca5c076740 tx=0x7fca5000a2d0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 msgr2=0x7fca5c074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 0x7fca5c074cb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca4c005940 tx=0x7fca4c004270 comp rx=0 tx=0).stop 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 shutdown_connections 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fca440b2960 0x7fca440b4e20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fca5c108980 0x7fca5c075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca5c103330 0x7fca5c0751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 --2- 172.21.15.3:0/2728007068 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fca5c102970 0x7fca5c074cb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 >> 172.21.15.3:0/2728007068 conn(0x7fca5c0fe460 msgr2=0x7fca5c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 shutdown_connections 2024-05-08T06:03:50.327 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.327+0000 7fca61ef8640 1 -- 172.21.15.3:0/2728007068 wait complete. 2024-05-08T06:03:50.338 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2071402009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104e20 msgr2=0x7f81f410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2071402009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104e20 0x7f81f410f950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f81ec009900 tx=0x7f81ec02a470 comp rx=0 tx=0).stop 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2071402009 shutdown_connections 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2071402009 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104e20 0x7f81f410f950 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2071402009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81f4104480 0x7f81f41048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2071402009 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f4102d40 0x7f81f4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.446 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2071402009 >> 172.21.15.3:0/2071402009 conn(0x7f81f40fe470 msgr2=0x7f81f4100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2071402009 shutdown_connections 2024-05-08T06:03:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.445+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2071402009 wait complete. 2024-05-08T06:03:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 Processor -- start 2024-05-08T06:03:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 -- start start 2024-05-08T06:03:50.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81f4102d40 0x7f81f41991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 0x7f81f4199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f4104e20 0x7f81f419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4112d40 con 0x7f81f4104480 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4112bc0 con 0x7f81f4104e20 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f8ae1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4112ec0 con 0x7f81f4102d40 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 0x7f81f4199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f1d9b640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 0x7f81f4199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40254/0 (socket says 172.21.15.3:40254) 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.446+0000 7f81f1d9b640 1 -- 172.21.15.3:0/2349831530 learned_addr learned my addr 172.21.15.3:0/2349831530 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f2d9d640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f4104e20 0x7f81f419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f259c640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81f4102d40 0x7f81f41991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f1d9b640 1 -- 172.21.15.3:0/2349831530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81f4102d40 msgr2=0x7f81f41991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f1d9b640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81f4102d40 0x7f81f41991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.449 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f1d9b640 1 -- 172.21.15.3:0/2349831530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f4104e20 msgr2=0x7f81f419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f1d9b640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f4104e20 0x7f81f419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f1d9b640 1 -- 172.21.15.3:0/2349831530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f419e1b0 con 0x7f81f4104480 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f1d9b640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 0x7f81f4199700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f81e800eb90 tx=0x7f81e800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8004d20 con 0x7f81f4104480 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f419e4a0 con 0x7f81f4104480 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e800ee90 con 0x7f81f4104480 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8010670 con 0x7f81f4104480 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.447+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41a5d30 con 0x7f81f4104480 2024-05-08T06:03:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.448+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f4069cb0 con 0x7f81f4104480 2024-05-08T06:03:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.450+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f81e8010870 con 0x7f81f4104480 2024-05-08T06:03:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.450+0000 7f81e37fe640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f81dc0b2960 0x7f81dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.450+0000 7f81f259c640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f81dc0b2960 0x7f81dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.451+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f81e80e3270 con 0x7f81f4104480 2024-05-08T06:03:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.451+0000 7f81f259c640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f81dc0b2960 0x7f81dc0b4e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009b60 tx=0x7f81e4005f90 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.454+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81e8054df0 con 0x7f81f4104480 2024-05-08T06:03:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.646+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f81f410afa0 con 0x7f81f4104480 2024-05-08T06:03:50.647 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.647+0000 7f81e37fe640 1 -- 172.21.15.3:0/2349831530 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) ==== 103+0+11 (secure 0 0 0) 0x7f81e80aa880 con 0x7f81f4104480 2024-05-08T06:03:50.647 INFO:teuthology.orchestra.run.smithi003.stdout:pg_num: 32 2024-05-08T06:03:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f81dc0b2960 msgr2=0x7f81dc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f81dc0b2960 0x7f81dc0b4e20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009b60 tx=0x7f81e4005f90 comp rx=0 tx=0).stop 2024-05-08T06:03:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 msgr2=0x7f81f4199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 0x7f81f4199700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f81e800eb90 tx=0x7f81e800c6a0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 shutdown_connections 2024-05-08T06:03:50.649 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f81dc0b2960 0x7f81dc0b4e20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f81f4104e20 0x7f81f419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f81f4104480 0x7f81f4199700 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 --2- 172.21.15.3:0/2349831530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81f4102d40 0x7f81f41991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 >> 172.21.15.3:0/2349831530 conn(0x7f81f40fe470 msgr2=0x7f81f4108890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 shutdown_connections 2024-05-08T06:03:50.650 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.649+0000 7f81f8ae1640 1 -- 172.21.15.3:0/2349831530 wait complete. 2024-05-08T06:03:50.660 DEBUG:teuthology.orchestra.run.smithi003:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 -- 172.21.15.3:0/2696416367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec01024a0 msgr2=0x7f3ec010e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/2696416367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec01024a0 0x7f3ec010e6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8009ca0 tx=0x7f3eb802a8c0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 -- 172.21.15.3:0/2696416367 shutdown_connections 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/2696416367 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec01024a0 0x7f3ec010e6d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/2696416367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec0101b00 0x7f3ec0101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/2696416367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ec0107930 0x7f3ec0107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 -- 172.21.15.3:0/2696416367 >> 172.21.15.3:0/2696416367 conn(0x7f3ec00786a0 msgr2=0x7f3ec0078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:50.767 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 -- 172.21.15.3:0/2696416367 shutdown_connections 2024-05-08T06:03:50.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 -- 172.21.15.3:0/2696416367 wait complete. 2024-05-08T06:03:50.768 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.767+0000 7f3ec52e1640 1 Processor -- start 2024-05-08T06:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 -- start start 2024-05-08T06:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 0x7f3ec019b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec01024a0 0x7f3ec019b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ec0107930 0x7f3ec019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.769 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0111bb0 con 0x7f3ec0101b00 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0111a30 con 0x7f3ec01024a0 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ec52e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec0111d30 con 0x7f3ec0107930 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 0x7f3ec019b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 0x7f3ec019b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40258/0 (socket says 172.21.15.3:40258) 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 -- 172.21.15.3:0/1208913868 learned_addr learned my addr 172.21.15.3:0/1208913868 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebf7fe640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ec0107930 0x7f3ec019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebe7fc640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec01024a0 0x7f3ec019b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 -- 172.21.15.3:0/1208913868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ec0107930 msgr2=0x7f3ec019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ec0107930 0x7f3ec019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 -- 172.21.15.3:0/1208913868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec01024a0 msgr2=0x7f3ec019b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec01024a0 0x7f3ec019b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 -- 172.21.15.3:0/1208913868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec01a02c0 con 0x7f3ec0101b00 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebe7fc640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec01024a0 0x7f3ec019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:03:50.770 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.768+0000 7f3ebeffd640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 0x7f3ec019b2d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e9f0 tx=0x7f3eb000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:50.773 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.773+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000cd80 con 0x7f3ec0101b00 2024-05-08T06:03:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.773+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eb00108e0 con 0x7f3ec0101b00 2024-05-08T06:03:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.773+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec01a05b0 con 0x7f3ec0101b00 2024-05-08T06:03:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.773+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0018980 con 0x7f3ec0101b00 2024-05-08T06:03:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.773+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec01a7e40 con 0x7f3ec0101b00 2024-05-08T06:03:50.774 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.775+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec0069860 con 0x7f3ec0101b00 2024-05-08T06:03:50.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.776+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3eb0010430 con 0x7f3ec0101b00 2024-05-08T06:03:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.777+0000 7f3ea7fff640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f3ea80b29e0 0x7f3ea80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:03:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.777+0000 7f3ebe7fc640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f3ea80b29e0 0x7f3ea80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:03:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.777+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9496+0+0 (secure 0 0 0) 0x7f3eb00e2cd0 con 0x7f3ec0101b00 2024-05-08T06:03:50.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.777+0000 7f3ebe7fc640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f3ea80b29e0 0x7f3ea80b4ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0101aa0 tx=0x7f3eb4005f90 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:03:50.781 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:50.782+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3eb00aa2e0 con 0x7f3ec0101b00 2024-05-08T06:03:50.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:50 smithi003 ceph-mon[30137]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2911058187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:03:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2728007068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:03:50.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/2349831530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T06:03:51.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:50 smithi104 ceph-mon[34559]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:51.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2911058187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:03:51.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2728007068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:03:51.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/2349831530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T06:03:51.021 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.022+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3ec01a80e0 con 0x7f3ec0101b00 2024-05-08T06:03:51.031 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.031+0000 7f3ea7fff640 1 -- 172.21.15.3:0/1208913868 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816164 (secure 0 0 0) 0x7f3eb00a9be0 con 0x7f3ec0101b00 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "epoch": 26, 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "active_gid": 15129, 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "active_name": "x", 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addrs": { 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6800", 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3346608619 2024-05-08T06:03:51.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v1", 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:6801", 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 3346608619 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "active_addr": "172.21.15.3:6801/3346608619", 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "active_change": "2024-05-08T05:53:46.504520+0000", 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "active_mgr_features": 4540701547738038271, 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "available": true, 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "standbys": [ 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24887, 2024-05-08T06:03:51.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "y", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.046 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-05-08T06:03:51.052 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-05-08T06:03:51.055 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-05-08T06:03:51.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.059 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.061 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.062 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-05-08T06:03:51.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.067 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.070 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.071 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.073 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-05-08T06:03:51.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.077 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.078 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.079 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.080 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.082 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-05-08T06:03:51.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.087 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-05-08T06:03:51.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.091 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.092 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.096 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-05-08T06:03:51.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.101 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.105 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.107 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:03:51.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-05-08T06:03:51.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-05-08T06:03:51.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.114 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.124 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.133 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-05-08T06:03:51.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.135 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-05-08T06:03:51.136 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.140 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.145 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.147 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.148 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.151 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:03:51.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.153 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.157 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.159 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:03:51.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.167 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.171 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "[]", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:03:51.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:03:51.174 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.178 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.182 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.185 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-05-08T06:03:51.187 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.190 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.193 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.194 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.195 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.196 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-05-08T06:03:51.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.201 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.203 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.206 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-05-08T06:03:51.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.209 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.219 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.220 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-05-08T06:03:51.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-05-08T06:03:51.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.223 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.226 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.227 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-05-08T06:03:51.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.230 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.232 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.235 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.236 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-05-08T06:03:51.238 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.240 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.241 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.243 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.254 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.255 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.256 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.257 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.259 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.260 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.263 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.264 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.265 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.267 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.268 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-05-08T06:03:51.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.270 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.271 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.272 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.273 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.274 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.278 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.280 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.282 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.285 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.286 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.288 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.289 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.290 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.292 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.293 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.295 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.296 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.297 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.298 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.299 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.301 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.303 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.305 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.307 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.308 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.309 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.312 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-05-08T06:03:51.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.315 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.317 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.319 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.321 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.326 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.327 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.329 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.331 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-05-08T06:03:51.334 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.335 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.338 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.339 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.341 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.344 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.345 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.346 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.347 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.348 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.351 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.355 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.357 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.358 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.360 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-08T06:03:51.362 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.363 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-05-08T06:03:51.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:03:51.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-08T06:03:51.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.374 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.376 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.378 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.382 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.384 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.385 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.386 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.387 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.388 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.390 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.394 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.395 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.398 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.400 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.401 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.404 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.405 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.406 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.407 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.409 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.410 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.412 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.414 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smb", 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.415 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.416 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.417 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.418 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "update_orchestration": { 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "update_orchestration", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-08T06:03:51.422 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.425 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.426 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.430 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.434 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.435 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.437 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.438 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.441 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.443 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.447 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.449 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.450 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-05-08T06:03:51.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-05-08T06:03:51.453 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-05-08T06:03:51.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-05-08T06:03:51.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.466 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.468 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.469 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.475 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.476 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.477 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.479 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.483 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.484 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.485 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.486 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.487 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.488 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.489 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:03:51.490 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.494 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.495 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.497 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.498 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.499 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.500 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.501 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.504 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "gid": 24892, 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "z", 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "mgr_features": 4540701547738038271, 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-05-08T06:03:51.505 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.507 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.509 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.510 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.511 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.512 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.513 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.514 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.515 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-05-08T06:03:51.516 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.517 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.518 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.519 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.520 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.522 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.523 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.524 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.525 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.526 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-05-08T06:03:51.527 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.528 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-08T06:03:51.529 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.530 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.531 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.532 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.533 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.534 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.535 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-05-08T06:03:51.536 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.537 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.538 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.539 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.540 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.541 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.542 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-05-08T06:03:51.544 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.545 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.546 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.547 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.548 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-05-08T06:03:51.549 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-05-08T06:03:51.550 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.551 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.552 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:03:51.553 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.554 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.555 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.556 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.557 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.558 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.559 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-05-08T06:03:51.560 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.561 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.562 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.563 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.564 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.565 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:03:51.566 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-05-08T06:03:51.567 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-05-08T06:03:51.568 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:03:51.569 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.570 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.571 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.572 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.573 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.574 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.575 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.576 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-05-08T06:03:51.577 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.578 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.579 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.580 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.581 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.582 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.583 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.584 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:03:51.585 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-05-08T06:03:51.586 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-05-08T06:03:51.587 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-05-08T06:03:51.588 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-08T06:03:51.589 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.590 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:03:51.591 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-05-08T06:03:51.592 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-05-08T06:03:51.593 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-05-08T06:03:51.594 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.595 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.596 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.597 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.598 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.599 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.600 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:03:51.601 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.602 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.603 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.604 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.605 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.606 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.607 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.608 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.609 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.610 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-05-08T06:03:51.611 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.612 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.613 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.614 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:03:51.615 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:03:51.616 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.617 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.618 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.619 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.620 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.621 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.622 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.623 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.624 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.625 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.626 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.627 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.628 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.629 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.630 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.631 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.632 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.633 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "[]", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.634 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.635 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.636 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.637 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.638 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.639 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.640 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.641 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.642 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.643 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.644 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.645 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.646 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.647 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.648 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.649 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.650 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.651 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.652 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:03:51.653 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:03:51.654 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.655 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.656 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.657 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.658 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.659 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.660 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.661 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.662 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.663 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.664 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.665 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.666 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:03:51.667 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:51.668 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.669 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.670 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.671 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.672 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.673 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.674 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.675 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.676 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.677 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.678 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.679 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.680 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.681 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.682 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-05-08T06:03:51.683 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-05-08T06:03:51.684 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.685 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-05-08T06:03:51.686 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.687 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.688 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.689 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.690 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.691 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.692 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.693 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.694 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.695 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.696 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.697 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.698 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.699 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.700 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.701 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.702 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-05-08T06:03:51.703 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.704 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-05-08T06:03:51.705 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.706 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.707 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.708 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.709 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.710 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.711 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.712 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.713 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.714 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.715 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.716 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.717 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.718 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.719 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.720 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.721 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.722 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.723 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.724 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.725 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.726 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.727 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.728 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.729 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.730 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-05-08T06:03:51.731 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.732 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.733 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.734 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.735 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.736 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.737 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.738 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.739 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:03:51.740 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.741 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.742 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.743 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.744 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.745 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.746 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-05-08T06:03:51.747 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.748 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.749 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.750 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:03:51.751 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.752 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.753 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.754 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.755 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.756 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.757 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.758 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.759 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.760 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.761 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.762 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.763 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.764 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.765 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.766 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.767 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.768 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.769 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.770 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.771 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.772 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.773 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.774 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.775 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.776 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.777 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.778 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.779 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.780 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.781 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.782 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.783 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-05-08T06:03:51.784 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.785 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.786 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.787 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.788 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.789 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.790 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.791 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.792 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.793 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.794 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.795 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.796 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.797 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.798 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.799 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.800 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.801 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.802 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.803 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.804 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.805 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.806 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.807 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.808 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.809 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-05-08T06:03:51.810 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.811 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.812 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.813 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.814 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.815 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.816 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.817 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.818 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.819 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:51.820 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:51.821 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.822 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.823 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.824 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-08T06:03:51.825 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.826 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.827 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.828 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.829 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.830 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.831 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.832 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.833 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.834 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.835 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.836 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.837 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.838 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.839 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.840 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.841 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.842 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.843 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.844 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.845 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.846 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.847 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.848 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.849 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.850 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.851 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.852 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.853 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.854 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.855 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.856 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.857 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.858 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.859 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.860 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.861 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.862 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.863 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.864 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.865 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.866 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.867 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.868 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.869 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.870 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.871 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.872 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.873 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.874 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.875 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.876 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.877 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smb", 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.878 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.879 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.880 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.881 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.882 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.883 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "update_orchestration": { 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "update_orchestration", 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.884 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-08T06:03:51.885 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:03:51.886 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.887 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.888 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.889 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.890 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.891 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.892 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.893 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.894 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.895 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.896 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.897 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.898 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.899 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.900 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.901 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.902 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.903 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.904 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.905 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.906 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.907 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.908 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.909 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.910 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.911 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.912 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.913 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.914 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.915 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.916 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.917 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.918 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.919 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.920 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.921 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.922 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.923 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.924 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.925 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.926 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.927 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.928 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.929 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.930 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.931 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.932 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-05-08T06:03:51.933 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-05-08T06:03:51.934 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.935 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.936 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.937 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.938 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.939 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.940 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.941 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.942 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.943 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.944 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.945 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.946 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.947 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.948 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.949 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.950 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.951 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.952 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.953 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.954 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.955 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.956 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.957 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.958 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.959 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.960 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.961 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.962 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.963 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.964 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:51.965 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.966 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "modules": [ 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard", 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "iostat", 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "nfs", 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "restful", 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "snap_schedule" 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "available_modules": [ 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.967 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "alerts", 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:51.968 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.969 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.970 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.971 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.972 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.973 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.974 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_destination": { 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_destination", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.975 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email address to send alerts to", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_from_name": { 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_from_name", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.976 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "Ceph", 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Email From: name", 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_host": { 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_host", 2024-05-08T06:03:51.977 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP server", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.978 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_password": { 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_password", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Password to authenticate with", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.979 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_port": { 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_port", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "465", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP port", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.980 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_sender": { 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_sender", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SMTP envelope sender", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.981 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_ssl": { 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_ssl", 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.982 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "smtp_user": { 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smtp_user", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.983 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "User to authenticate as", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.984 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "balancer", 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:51.985 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "active": { 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "active", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T06:03:51.986 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_time": { 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_time", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0000", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.987 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "begin_weekday": { 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "begin_weekday", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-08T06:03:51.988 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_max_iterations": { 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_max_iterations", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "25", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "250", 2024-05-08T06:03:51.989 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_metrics": { 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_metrics", 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T06:03:51.990 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "crush_compat_step": { 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crush_compat_step", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.991 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.5", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0.001", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "0.999", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "aggressiveness of optimization", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "end_time": { 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_time", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.992 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2359", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "end_weekday": { 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "end_weekday", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:51.993 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "0", 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "6", 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:51.994 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.995 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.996 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:51.997 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:51.998 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:51.999 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "min_score": { 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_score", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:52.000 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-08T06:03:52.001 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "upmap", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "crush-compat", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "none", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "read", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap", 2024-05-08T06:03:52.002 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap-read" 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Balancer mode", 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_ids": { 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_ids", 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.003 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.004 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.005 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_deviation": { 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_deviation", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.006 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "upmap_max_optimizations": { 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "upmap_max_optimizations", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:52.007 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.008 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_down_multiplier": { 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_down_multiplier", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.009 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_refresh_rate": { 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_refresh_rate", 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "20", 2024-05-08T06:03:52.010 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "agent_starting_port": { 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "agent_starting_port", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.011 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4721", 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_ptrace": { 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_ptrace", 2024-05-08T06:03:52.012 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.013 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_interval": { 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_interval", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.014 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "autotune_memory_target_ratio": { 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0.7", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.015 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm_log_destination": { 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cephadm_log_destination", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "file", 2024-05-08T06:03:52.016 INFO:teuthology.orchestra.run.smithi003.stdout: "file,syslog", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "syslog" 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "cgroups_split": { 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cgroups_split", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.017 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "config_checks_enabled": { 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_checks_enabled", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.018 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "config_dashboard": { 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_dashboard", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.019 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_alertmanager": { 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_alertmanager", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.020 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_base": { 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_base", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.021 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Container image name, without the tag", 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.022 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_elasticsearch": { 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_elasticsearch", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "elasticsearch container image", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.023 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_grafana": { 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_grafana", 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.024 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_haproxy": { 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_haproxy", 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T06:03:52.025 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "HAproxy container image", 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_agent": { 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_agent", 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.026 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger agent container image", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_collector": { 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_collector", 2024-05-08T06:03:52.027 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger collector container image", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_jaeger_query": { 2024-05-08T06:03:52.028 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_jaeger_query", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Jaeger query container image", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.029 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_keepalived": { 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_keepalived", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Keepalived container image", 2024-05-08T06:03:52.030 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_loki": { 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_loki", 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.031 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Loki container image", 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.032 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_node_exporter": { 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_node_exporter", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.033 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_nvmeof": { 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_nvmeof", 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.034 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Nvme-of container image", 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_prometheus": { 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_prometheus", 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.035 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Prometheus container image", 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.036 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_promtail": { 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_promtail", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Promtail container image", 2024-05-08T06:03:52.037 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_samba": { 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_samba", 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.038 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Samba/SMB container image", 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "container_image_snmp_gateway": { 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_image_snmp_gateway", 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.039 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "SNMP Gateway container image", 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "container_init": { 2024-05-08T06:03:52.040 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "container_init", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.041 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "daemon_cache_timeout": { 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "daemon_cache_timeout", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.042 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_cephadm_command_timeout": { 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "900", 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.043 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_registry": { 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "default_registry", 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "docker.io", 2024-05-08T06:03:52.044 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "device_cache_timeout": { 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_cache_timeout", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1800", 2024-05-08T06:03:52.045 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache device inventory", 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "device_enhanced_scan": { 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_enhanced_scan", 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.046 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "facts_cache_timeout": { 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "facts_cache_timeout", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.047 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "seconds to cache host facts data", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "grafana_dashboards_path": { 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "grafana_dashboards_path", 2024-05-08T06:03:52.048 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "host_check_interval": { 2024-05-08T06:03:52.049 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "host_check_interval", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to perform a host check", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.050 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "hw_monitoring": { 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hw_monitoring", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.051 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "inventory_list_all": { 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "inventory_list_all", 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.052 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.053 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.054 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "log_refresh_metadata": { 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_refresh_metadata", 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.055 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.056 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.057 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.058 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.059 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.060 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "*", 2024-05-08T06:03:52.061 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "max_count_per_host": { 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_count_per_host", 2024-05-08T06:03:52.062 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of daemons per service per host", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "max_osd_draining_count": { 2024-05-08T06:03:52.063 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_osd_draining_count", 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.064 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "migration_current": { 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "migration_current", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.065 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "internal - do not modify", 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "mode": { 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mode", 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "root", 2024-05-08T06:03:52.066 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm-package", 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "root" 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "oob_default_addr": { 2024-05-08T06:03:52.067 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "oob_default_addr", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "169.254.1.1", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.068 INFO:teuthology.orchestra.run.smithi003.stdout: "prometheus_alerts_path": { 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus_alerts_path", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.069 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_insecure": { 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_insecure", 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.070 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_password": { 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_password", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.071 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_url": { 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_url", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.072 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "registry_username": { 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "registry_username", 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.073 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "secure_monitoring_stack": { 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "secure_monitoring_stack", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.074 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "service_discovery_port": { 2024-05-08T06:03:52.075 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "service_discovery_port", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8765", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "cephadm service discovery port", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.076 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.077 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "ssh_config_file": { 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssh_config_file", 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.078 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "use_agent": { 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_agent", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.079 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "use_repo_digest": { 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "use_repo_digest", 2024-05-08T06:03:52.080 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.081 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_failed_host_check": { 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_failed_host_check", 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.082 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_daemons": { 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_daemons", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.083 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_on_stray_hosts": { 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_on_stray_hosts", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.084 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.085 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crash", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.086 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.087 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.088 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.089 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.090 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.091 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "retain_interval": { 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retain_interval", 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "31536000", 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.092 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.093 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_recent_interval": { 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_recent_interval", 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.094 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1209600", 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.095 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dashboard", 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.096 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.097 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T06:03:52.098 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_ENABLED": { 2024-05-08T06:03:52.099 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.100 INFO:teuthology.orchestra.run.smithi003.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.101 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.102 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.103 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.104 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.105 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T06:03:52.106 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.107 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.108 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.109 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.110 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T06:03:52.111 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.112 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.113 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.114 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_URL": { 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_URL", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.115 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.116 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.117 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.118 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T06:03:52.119 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.120 INFO:teuthology.orchestra.run.smithi003.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.121 INFO:teuthology.orchestra.run.smithi003.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "[]", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.122 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "{}", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.123 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.124 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.125 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.126 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.127 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.128 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T06:03:52.129 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.130 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.131 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.132 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.133 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.134 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.135 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.136 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.137 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8", 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.138 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "45", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.139 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.140 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.141 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "admin", 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.143 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.144 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.145 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.146 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.147 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.148 INFO:teuthology.orchestra.run.smithi003.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.149 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "cross_origin_url": { 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cross_origin_url", 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.150 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "crt_file": { 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "crt_file", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.151 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "debug": { 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "debug", 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.152 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Enable/disable debug options", 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "jwt_token_ttl": { 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "jwt_token_ttl", 2024-05-08T06:03:52.153 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "28800", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.154 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.155 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.156 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.157 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.158 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.159 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.160 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "motd": { 2024-05-08T06:03:52.161 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "motd", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "The message of the day", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.162 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect_resolve_ip_addr": { 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.163 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.164 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8080", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.165 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.166 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.167 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl_server_port": { 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl_server_port", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.168 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8443", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.169 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "redirect", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "redirect" 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.170 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.171 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "url_prefix": { 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url_prefix", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.172 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_monitoring": { 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_monitoring", 2024-05-08T06:03:52.173 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "monitor device health metrics", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.174 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.175 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.176 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.177 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.178 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.179 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "mark_out_threshold": { 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mark_out_threshold", 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.180 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "2419200", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "pool_name": { 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pool_name", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.181 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "device_health_metrics", 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "retention_period": { 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "retention_period", 2024-05-08T06:03:52.182 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15552000", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long to retain device health metrics", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.183 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_frequency": { 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_frequency", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.184 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "self_heal": { 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "self_heal", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T06:03:52.185 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.186 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.187 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "warn_threshold": { 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "warn_threshold", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.188 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7257600", 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.189 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "diskprediction_local", 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.190 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.191 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.192 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.193 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.194 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.195 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "predict_interval": { 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predict_interval", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "86400", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.196 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "predictor_model": { 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "predictor_model", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "prophetstor", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.197 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "600", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.198 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.199 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "influx", 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": false, 2024-05-08T06:03:52.200 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "influxdb python module not found", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "batch_size": { 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "batch_size", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5000", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T06:03:52.201 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "database": { 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "database", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "ceph", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.202 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "hostname": { 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "hostname", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.203 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server hostname", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "30", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "5", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.204 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.205 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.206 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.207 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.208 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.209 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.210 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "password": { 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "password", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "password of InfluxDB server user", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.211 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "port": { 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "port", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "8086", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "InfluxDB server port", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.212 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.213 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "ssl": { 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ssl", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "false", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:03:52.214 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "threads": { 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threads", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "32", 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.215 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "username": { 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "username", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.216 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "username of InfluxDB server user", 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "verify_ssl": { 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "verify_ssl", 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.217 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "true", 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.218 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "insights", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.219 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.220 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.221 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.222 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.223 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.224 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.225 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "iostat", 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.226 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.227 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.228 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.229 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.230 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.231 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.232 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "k8sevents", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "ceph_event_retention_days": { 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "ceph_event_retention_days", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "7", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.233 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "config_check_secs": { 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "config_check_secs", 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.234 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "10", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.235 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.236 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.237 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.238 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.239 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.240 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.241 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "localpool", 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "failure_domain": { 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "failure_domain", 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "host", 2024-05-08T06:03:52.242 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "failure domain for any created local pool", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.243 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.244 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.245 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.246 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.247 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.248 INFO:teuthology.orchestra.run.smithi003.stdout: "min_size": { 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "min_size", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default min_size for any created local pool", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.249 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "num_rep": { 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "num_rep", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default replica count for any created local pool", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.250 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_num": { 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_num", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "128", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.251 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "prefix": { 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prefix", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "name prefix for any created local pool", 2024-05-08T06:03:52.252 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.253 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "subtree": { 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "subtree", 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "rack", 2024-05-08T06:03:52.254 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mds_autoscaler", 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.255 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.256 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.257 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.258 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.259 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.260 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.261 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.262 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirroring", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.263 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.264 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.265 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.266 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.267 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.268 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.269 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "nfs", 2024-05-08T06:03:52.270 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.271 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.272 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.273 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.274 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.275 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.276 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.277 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "fail_fs": { 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "fail_fs", 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.278 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.279 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.280 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.281 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.282 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.283 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.284 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator": { 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "orchestrator", 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.285 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "cephadm", 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "rook", 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "test_orchestrator" 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Orchestrator backend", 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.286 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.287 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_perf_query", 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.288 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.289 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.290 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.291 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.292 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.293 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.294 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "osd_support", 2024-05-08T06:03:52.295 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.296 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.297 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.298 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.299 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.300 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.301 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.302 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "pg_autoscaler", 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.303 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.304 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.305 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.306 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.307 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.308 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:52.309 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.310 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "threshold": { 2024-05-08T06:03:52.311 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "threshold", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "3.0", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1.0", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "scaling threshold", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.312 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "progress", 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_pg_recovery_event": { 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_pg_recovery_event", 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.313 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-08T06:03:52.314 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.315 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.316 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.317 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.318 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.319 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.320 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "max_completed_events": { 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_completed_events", 2024-05-08T06:03:52.321 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "50", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "number of past completed events to remember", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.322 INFO:teuthology.orchestra.run.smithi003.stdout: "sleep_interval": { 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sleep_interval", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "5", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "how long the module is going to sleep", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.323 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.324 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "prometheus", 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.325 INFO:teuthology.orchestra.run.smithi003.stdout: "cache": { 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "cache", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.326 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "exclude_perf_counters": { 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "exclude_perf_counters", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T06:03:52.327 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.328 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.329 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.330 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.331 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.332 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools": { 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.333 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T06:03:52.334 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "300", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.335 INFO:teuthology.orchestra.run.smithi003.stdout: "scrape_interval": { 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "scrape_interval", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15.0", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "::", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.337 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "9283", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.338 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.339 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "stale_cache_strategy": { 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stale_cache_strategy", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "log", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.340 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_behaviour": { 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_behaviour", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "default", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.341 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "default", 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "error" 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "standby_error_status_code": { 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "standby_error_status_code", 2024-05-08T06:03:52.342 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "500", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "400", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "599", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.343 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.344 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.345 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.346 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.347 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.348 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.349 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_snap_create": { 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_snap_create", 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.350 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "mirror_snapshot_schedule": { 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.351 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.352 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "trash_purge_schedule": { 2024-05-08T06:03:52.353 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "trash_purge_schedule", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.354 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "restful", 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "enable_auth": { 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enable_auth", 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.355 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "key_file": { 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "key_file", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.356 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.357 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.358 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.359 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.360 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.361 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.362 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "server_addr": { 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_addr", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.363 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "server_port": { 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "server_port", 2024-05-08T06:03:52.364 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.365 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.366 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rgw", 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.367 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.368 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.369 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.370 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.371 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.372 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.373 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.374 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rook", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.375 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.376 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.377 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.378 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.379 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.380 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.381 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "storage_class": { 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "storage_class", 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "local", 2024-05-08T06:03:52.382 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "selftest", 2024-05-08T06:03:52.383 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.384 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.385 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.386 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.387 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.388 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "roption1": { 2024-05-08T06:03:52.389 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption1", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.390 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "roption2": { 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "roption2", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "xyz", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.391 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption1": { 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption1", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.392 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption2": { 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption2", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.393 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption3": { 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption3", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "float", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.394 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption4": { 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption4", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.395 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption5": { 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption5", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.396 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption6": { 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption6", 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.397 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.398 INFO:teuthology.orchestra.run.smithi003.stdout: "rwoption7": { 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rwoption7", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "1", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "42", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.399 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.400 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "testkey": { 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testkey", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.401 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "testlkey": { 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testlkey", 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.402 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "testnewline": { 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "testnewline", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.403 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.404 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "smb", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.405 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.406 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.407 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.408 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.409 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.410 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.411 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "update_orchestration": { 2024-05-08T06:03:52.412 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "update_orchestration", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.413 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "allow_m_granularity": { 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "allow_m_granularity", 2024-05-08T06:03:52.414 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.417 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "dump_on_update": { 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "dump_on_update", 2024-05-08T06:03:52.418 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "dump database to debug log on update", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.419 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.420 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.421 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.422 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.423 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.424 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.425 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "stats", 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.426 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.427 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.428 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.429 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.430 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.431 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.432 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "status", 2024-05-08T06:03:52.433 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.434 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.435 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.436 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.437 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.438 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.439 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.440 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telegraf", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "address": { 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "address", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.441 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "15", 2024-05-08T06:03:52.442 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.443 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.444 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.445 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.446 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.447 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.448 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.449 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "telemetry", 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_basic": { 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_basic", 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.450 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_crash": { 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_crash", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.451 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_device": { 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_device", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.452 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.453 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_ident": { 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_ident", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.454 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "channel_perf": { 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "channel_perf", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T06:03:52.455 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "contact": { 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "contact", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.456 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "description": { 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "description", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.457 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "device_url": { 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "device_url", 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T06:03:52.458 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "enabled": { 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "enabled", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.459 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:52.460 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "24", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "8", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "last_opt_revision": { 2024-05-08T06:03:52.461 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "last_opt_revision", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "1", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.462 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard": { 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.463 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "leaderboard_description": { 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "leaderboard_description", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.464 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.465 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.466 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.467 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.468 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.469 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.470 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "organization": { 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "organization", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.471 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "proxy": { 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "proxy", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.472 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.473 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "url": { 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "url", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.474 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.475 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "test_orchestrator", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.476 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.477 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.478 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.479 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.480 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.481 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.482 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.483 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.484 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.485 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.486 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.487 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.488 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "max_concurrent_clones": { 2024-05-08T06:03:52.489 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "max_concurrent_clones", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "4", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.490 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "periodic_async_work": { 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "periodic_async_work", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Periodically check for async work", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.491 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_delay": { 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_delay", 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.492 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "snapshot_clone_no_wait": { 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "True", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.493 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.494 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.495 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "can_run": true, 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "error_string": "", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "module_options": { 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "discovery_interval": { 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "discovery_interval", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "uint", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "100", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.496 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "identifier": { 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "identifier", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.497 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "interval": { 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "interval", 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "secs", 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.498 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "60", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "log_level": { 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_level", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.499 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.500 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster": { 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster", 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.501 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_cluster_level": { 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_cluster_level", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.502 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "info", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [ 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "critical", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "debug", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "error", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "info", 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: "warning" 2024-05-08T06:03:52.503 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "log_to_file": { 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "log_to_file", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "bool", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "False", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.504 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "sqlite3_killpoint": { 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "sqlite3_killpoint", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "dev", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 1, 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "0", 2024-05-08T06:03:52.505 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_host": { 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_host", 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.506 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "", 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_port": { 2024-05-08T06:03:52.507 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_port", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "int", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "10051", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.508 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "zabbix_sender": { 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "zabbix_sender", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "str", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "level": "advanced", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "flags": 0, 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "min": "", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "max": "", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "enum_allowed": [], 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "desc": "", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "long_desc": "", 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "tags": [], 2024-05-08T06:03:52.509 INFO:teuthology.orchestra.run.smithi003.stdout: "see_also": [] 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "services": { 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "dashboard": "https://172.21.15.3:8443/" 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "always_on_modules": { 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "octopus": [ 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-08T06:03:52.510 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "pacific": [ 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-08T06:03:52.511 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "quincy": [ 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-08T06:03:52.512 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "reef": [ 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-08T06:03:52.513 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: ], 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "squid": [ 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "balancer", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "crash", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "devicehealth", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "orchestrator", 2024-05-08T06:03:52.514 INFO:teuthology.orchestra.run.smithi003.stdout: "pg_autoscaler", 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "progress", 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "rbd_support", 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "status", 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "telemetry", 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "volumes" 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "last_failure_osd_epoch": 90, 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "active_clients": [ 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "devicehealth", 2024-05-08T06:03:52.515 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.516 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-05-08T06:03:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2167843214 2024-05-08T06:03:52.517 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "libcephsqlite", 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 1223886891 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.518 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "rbd_support", 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2071262549 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.519 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2131023878 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "snap_schedule", 2024-05-08T06:03:52.520 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 110703964 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "name": "volumes", 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "addrvec": [ 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: { 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "type": "v2", 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "addr": "172.21.15.3:0", 2024-05-08T06:03:52.521 INFO:teuthology.orchestra.run.smithi003.stdout: "nonce": 2336942988 2024-05-08T06:03:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:52.522 INFO:teuthology.orchestra.run.smithi003.stdout: ] 2024-05-08T06:03:52.522 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T06:03:52.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:51 smithi104 ceph-mon[34559]: pgmap v314: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:52.523 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:51 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1208913868' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:03:52.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:50 smithi089 ceph-mon[34495]: from='client.15657 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T06:03:52.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2911058187' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T06:03:52.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2728007068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T06:03:52.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/2349831530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T06:03:52.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:51 smithi089 ceph-mon[34495]: pgmap v314: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:52.524 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:51 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1208913868' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:03:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f3ea80b29e0 msgr2=0x7f3ea80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.525 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f3ea80b29e0 0x7f3ea80b4ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0101aa0 tx=0x7f3eb4005f90 comp rx=0 tx=0).stop 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 msgr2=0x7f3ec019b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 0x7f3ec019b2d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e9f0 tx=0x7f3eb000eeb0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 shutdown_connections 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f3ea80b29e0 0x7f3ea80b4ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ec0107930 0x7f3ec019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3ec01024a0 0x7f3ec019b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 --2- 172.21.15.3:0/1208913868 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f3ec0101b00 0x7f3ec019b2d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.036+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 >> 172.21.15.3:0/1208913868 conn(0x7f3ec00786a0 msgr2=0x7f3ec0069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.037+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 shutdown_connections 2024-05-08T06:03:52.526 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:03:51.037+0000 7f3ec52e1640 1 -- 172.21.15.3:0/1208913868 wait complete. 2024-05-08T06:03:52.533 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T06:03:52.534 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T06:03:52.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:51 smithi003 ceph-mon[30137]: pgmap v314: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:03:52.534 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:51 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1208913868' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T06:03:52.799 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:54.064 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:53 smithi089 ceph-mon[34495]: pgmap v315: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:54.089 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:53 smithi003 ceph-mon[30137]: pgmap v315: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:53 smithi104 ceph-mon[34559]: pgmap v315: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:54.785 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 4, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 4, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 2944, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 448, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001906852, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000040621, 2024-05-08T06:03:54.786 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000736591, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 3, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000754904, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000251634 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.787 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 239, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 228, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1715360, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 140704, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.049742975, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.015640430, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.038812079, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.004803012, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 227, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.011694295, 2024-05-08T06:03:54.788 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000051516 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1715360, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 140704 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 175, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 187968, 2024-05-08T06:03:54.789 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.015563339, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.010253081, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.002671726, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 172, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.016186305, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000094106 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:54.790 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 187968 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:03:54.791 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:03:54.792 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:03:54.793 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148732, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:03:54.794 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 241, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1681627, 2024-05-08T06:03:54.795 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 241, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 241, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1681627, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:54.796 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T06:03:55.552 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-08T06:03:55.552 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-08T06:03:55.552 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-08T06:03:55.552 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T06:03:55.553 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T06:03:55.813 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:55.842 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:55 smithi003 ceph-mon[30137]: pgmap v316: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:56.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:55 smithi089 ceph-mon[34495]: pgmap v316: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:56.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:55 smithi104 ceph-mon[34559]: pgmap v316: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-05-08T06:03:57.540 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001445091, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000212312, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:03:57.541 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 179, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 188672, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.015291558, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.010824360, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001890418, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 176, 2024-05-08T06:03:57.542 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.015856345, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000090092 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 188672 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 250, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 237, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1720272, 2024-05-08T06:03:57.543 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 145152, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.052008889, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.016249349, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.040733336, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.004813877, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 237, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.012470049, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000052616 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:57.544 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1720272, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 145152 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:03:57.545 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-05-08T06:03:57.546 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-05-08T06:03:57.547 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148731, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-05-08T06:03:57.548 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-05-08T06:03:57.549 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 248, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1682601, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 248, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 248, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1682601, 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-05-08T06:03:57.550 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:03:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:03:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:03:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:57.551 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:03:57.551 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T06:03:57.878 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:57 smithi003 ceph-mon[30137]: pgmap v317: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:58.212 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-08T06:03:58.212 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-08T06:03:58.212 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-08T06:03:58.212 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T06:03:58.212 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T06:03:58.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:57 smithi089 ceph-mon[34495]: pgmap v317: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:58.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:57 smithi104 ceph-mon[34559]: pgmap v317: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:03:58.471 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:03:59.900 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:03:59 smithi003 ceph-mon[30137]: pgmap v318: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:04:00.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:03:59 smithi089 ceph-mon[34495]: pgmap v318: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:04:00.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:03:59 smithi104 ceph-mon[34559]: pgmap v318: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:04:00.331 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T06:04:00.331 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.001277685, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000190142, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:00.332 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 177, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:04:00.333 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 188320, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.014957714, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.010602746, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001877649, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 174, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.017094739, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000098245 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:00.334 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 188320 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 245, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 232, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1718848, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 142432, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.052405034, 2024-05-08T06:04:00.335 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.016864400, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.040761001, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.005032710, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 232, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.012675275, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000054634 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1718848, 2024-05-08T06:04:00.336 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 142432 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:00.337 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:00.338 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:00.339 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148731, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 31, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:00.340 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:00.341 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 243, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1681887, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 243, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 243, 2024-05-08T06:04:00.342 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1681887, 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:04:00.343 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T06:04:01.057 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-08T06:04:01.057 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-08T06:04:01.057 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-08T06:04:01.057 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T06:04:01.057 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T06:04:01.328 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:02.153 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:04:02.153 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:02.153 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.153 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.154 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.154 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.154 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.154 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:01 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:01 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-08T06:04:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T06:04:02.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:02.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:01 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:03.136 INFO:teuthology.orchestra.run.smithi003.stdout:{ 2024-05-08T06:04:03.136 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 253, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 240, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 1721088, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 147072, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.048968414, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.015674993, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.036765289, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.004944406, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 240, 2024-05-08T06:04:03.137 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.012271830, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000051132 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 1721088, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 147072 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 2, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 182, 2024-05-08T06:04:03.138 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 189200, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.015856056, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.011341463, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.001852580, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 179, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.017742618, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000099120 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.139 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 189200 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:03.140 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_total_time": 0.000292612, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_recv_time": 0.000216538, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.141 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "cct": { 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "total_workers": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "unhealthy_workers": 0 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "mempool": { 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:03.142 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:03.143 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_bytes": 148714, 2024-05-08T06:04:03.144 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_items": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_items": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:03.145 INFO:teuthology.orchestra.run.smithi003.stdout: "pgmap_items": 0, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "mds_co_items": 2, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_1_items": 0, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "unittest_2_items": 0 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: }, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "val": 0, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "max": 104857600, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get_started": 0, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get": 251, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get_sum": 1682991, 2024-05-08T06:04:03.146 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "get_or_fail_success": 251, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "take": 0, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "take_sum": 0, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "put": 251, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "put_sum": 1682991, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "wait": { 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "avgcount": 0, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "sum": 0.000000000, 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: "avgtime": 0.000000000 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout: } 2024-05-08T06:04:03.147 INFO:teuthology.orchestra.run.smithi003.stdout:} 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:03.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:03.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:03 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-08T06:04:04.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:03 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:03 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.580 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-08T06:04:04.580 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-08T06:04:04.580 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-08T06:04:04.580 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T06:04:04.581 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T06:04:04.823 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:04.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi089 to 176.1M 2024-05-08T06:04:04.911 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:04 smithi003 ceph-mon[30137]: Unable to set osd_memory_target on smithi089 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-08T06:04:04.945 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-05-08T06:04:04.945 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T06:04:04.945 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.945 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:04.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi089 to 176.1M 2024-05-08T06:04:04.946 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:04 smithi089 ceph-mon[34495]: Unable to set osd_memory_target on smithi089 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi104 to 176.2M 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: Unable to set osd_memory_target on smithi104 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi089 to 176.1M 2024-05-08T06:04:05.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:04 smithi104 ceph-mon[34559]: Unable to set osd_memory_target on smithi089 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: pgmap v321: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:06.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:06.014 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:05 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: pgmap v321: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:06.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:05 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: pgmap v321: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T06:04:06.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:05 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' 2024-05-08T06:04:06.615 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-05-08T06:04:06.615 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:06.615 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 1463, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 1461, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1998004, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 509638, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.138269130, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.054627798, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.036830673, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.036455649, 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:06.616 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1460, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.054442104, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000037289 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 367632, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 100368 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.617 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 9, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 128, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 6169, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 301722, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 4, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.007289300, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.004603816, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001419722, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000178972, 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:06.618 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 127, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.010479129, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000082512 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 300384 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 594, 2024-05-08T06:04:06.619 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 596, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 146329, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 188899, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.050439890, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.014178568, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.011790117, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.020063146, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 594, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.018168192, 2024-05-08T06:04:06.620 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000030586 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 1, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-MDSRank": { 2024-05-08T06:04:06.621 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3168, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.078725134, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000024850 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "queue_len": 0, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "complete_latency": { 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.622 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "mds": { 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "request": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "reply": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "reply_latency": { 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_reply": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "forward": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:06.623 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_commit": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_split": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_merge": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes": 24, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_top": 10, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_bottom": 14, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pin_tail": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_pinned": 14, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_expired": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "inodes_with_caps": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "caps": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "subtrees": 2, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse": 0, 2024-05-08T06:04:06.624 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_hit": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_forward": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_discover": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "traverse_lock": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "load_cent": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "q": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "exported": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "exported_inodes": 0, 2024-05-08T06:04:06.625 INFO:teuthology.orchestra.run.smithi089.stdout: "imported": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "imported_inodes": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rfiles": 4, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rbytes": 234, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "root_rsnaps": 6, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:06.626 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "scrub_file_inodes": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_grant": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:04:06.627 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_caps_dirty": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_cap_release": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "process_request_cap_release": 0 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_cache": { 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_try_discover": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_send_discover": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "dir_handle_discover": 0, 2024-05-08T06:04:06.628 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_created": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_enqueued": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "strays_migrated": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_started": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "recovery_completed": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_path": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_quiesce_inode": 0, 2024-05-08T06:04:06.629 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_exportdir": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_flush": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ireq_inodestats": 0 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_log": { 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "evlrg": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "evadd": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "evex": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "evtrm": 0, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "ev": 237, 2024-05-08T06:04:06.630 INFO:teuthology.orchestra.run.smithi089.stdout: "evexg": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "evexd": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "segadd": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "segex": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "segtrm": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "seg": 2, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "segmjr": 13, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "segexg": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "segexd": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "expos": 5412460, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "wrpos": 5517448, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "rdpos": 5517364, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "jlat": { 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.631 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "replayed": 237 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_mem": { 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "ino": 15, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "ino+": 19, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "ino-": 4, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "dir": 10, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "dir+": 10, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "dir-": 0, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "dn": 24, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "dn+": 24, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "dn-": 0, 2024-05-08T06:04:06.632 INFO:teuthology.orchestra.run.smithi089.stdout: "cap": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "cap+": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "cap-": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "rss": 50060, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "heap": 215416 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_server": { 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_client_request": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "dispatch_server_request": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_request": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_client_session": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "handle_peer_request": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "req_create_latency": { 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.633 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getattr_latency": { 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "req_link_latency": { 2024-05-08T06:04:06.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookup_latency": { 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.635 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupino_latency": { 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupname_latency": { 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.636 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "req_lssnap_latency": { 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mkdir_latency": { 2024-05-08T06:04:06.637 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.638 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.638 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.638 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.638 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mknod_latency": { 2024-05-08T06:04:06.638 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "req_mksnap_latency": { 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "req_open_latency": { 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.639 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "req_readdir_latency": { 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rename_latency": { 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.640 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmdir_latency": { 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.641 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setattr_latency": { 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:06.642 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setlayout_latency": { 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.643 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "req_setxattr_latency": { 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "req_symlink_latency": { 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "req_unlink_latency": { 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.644 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_sessions": { 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "session_count": 2, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "session_add": 4, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "session_remove": 2, 2024-05-08T06:04:06.645 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_open": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "sessions_stale": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "total_load": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "average_load": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "avg_session_uptime": 3599, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:06.646 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:06.647 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:06.648 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 189716, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 59, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 9512, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 182, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:06.649 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 80096, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 666, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "objecter": { 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_active": 0, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_laggy": 0, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send": 1885, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_send_bytes": 26, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_resend": 4, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_reply": 1885, 2024-05-08T06:04:06.650 INFO:teuthology.orchestra.run.smithi089.stdout: "op_latency": { 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1885, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.798983367, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000423863 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "op_inflight": 0, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "oplen_avg": { 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 1885, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 1886 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "op": 1885, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "op_r": 1885, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "op_w": 0, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "op_rmw": 0, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "op_pg": 0, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_stat": 1182, 2024-05-08T06:04:06.651 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_create": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_read": 700, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_write": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writefull": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_writesame": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_append": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_zero": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_truncate": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_delete": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_mapext": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_getxattr": 2, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_setxattr": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_call": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_watch": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_notify": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "osdop_other": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_active": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_send": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_resend": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "linger_ping": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_active": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_send": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "poolop_resend": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_active": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_send": 0, 2024-05-08T06:04:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: "poolstat_resend": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_active": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_send": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "statfs_resend": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "command_active": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "command_send": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "command_resend": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "map_epoch": 99, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "map_full": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "map_inc": 8, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_sessions": 5, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_open": 5, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_session_close": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_laggy": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_wr": 0, 2024-05-08T06:04:06.654 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_rd": 2, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_del": 0 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "oft": { 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_objs": 0, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_updates": 0, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "omap_total_removes": 0 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "osd-slow-ops": { 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "slow_ops_count": 0 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "purge_queue": { 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing": 0, 2024-05-08T06:04:06.655 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executing_high_water": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed_ops": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_executed": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 2062, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1952961, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 2062, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 2062, 2024-05-08T06:04:06.656 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1952961, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:06.657 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 1885, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1250465, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 1885, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1250465, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 1024, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-05-08T06:04:06.658 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 1885, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 1885, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 1885, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1885, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:06.659 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.660 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-write_buf_throttle-0x55bf14a1e6e0": { 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 3758096384, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 0, 2024-05-08T06:04:06.661 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 0, 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:06.662 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-05-08T06:04:07.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:06 smithi089 ceph-mon[34495]: Adjusting osd_memory_target on smithi003 to 177.2M 2024-05-08T06:04:07.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:06 smithi089 ceph-mon[34495]: Unable to set osd_memory_target on smithi003 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-08T06:04:07.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:06 smithi003 ceph-mon[30137]: Adjusting osd_memory_target on smithi003 to 177.2M 2024-05-08T06:04:07.161 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:06 smithi003 ceph-mon[30137]: Unable to set osd_memory_target on smithi003 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-08T06:04:07.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:06 smithi104 ceph-mon[34559]: Adjusting osd_memory_target on smithi003 to 177.2M 2024-05-08T06:04:07.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:06 smithi104 ceph-mon[34559]: Unable to set osd_memory_target on smithi003 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-08T06:04:07.414 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-08T06:04:07.414 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-05-08T06:04:07.414 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T06:04:07.415 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T06:04:07.415 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T06:04:07.664 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:08.013 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:07 smithi089 ceph-mon[34495]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:04:08.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:07 smithi003 ceph-mon[30137]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:04:08.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:07 smithi104 ceph-mon[34559]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 248, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 237, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1717872, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 145984, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.052750550, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.015552206, 2024-05-08T06:04:09.297 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.040284312, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.006279351, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 236, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.013140659, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000055680 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.298 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1717872, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 145984 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 2, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 182, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 189200, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.016071534, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008624070, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.001271305, 2024-05-08T06:04:09.299 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 179, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.019517538, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000109036 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:09.300 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 189200 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 4, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 4, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 2944, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 448, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000585657, 2024-05-08T06:04:09.301 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000032740, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000556815, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 3, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000773694, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000257898 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:09.302 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-05-08T06:04:09.303 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:09.304 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:09.305 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148699, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:09.306 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:09.307 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 250, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1682861, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 250, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-05-08T06:04:09.308 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 250, 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1682861, 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:09.309 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-05-08T06:04:09.951 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-05-08T06:04:09.951 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-05-08T06:04:09.951 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-05-08T06:04:09.951 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T06:04:09.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T06:04:10.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:09 smithi003 ceph-mon[30137]: pgmap v323: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:04:10.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:09 smithi003 ceph-mon[30137]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T06:04:10.191 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:09 smithi089 ceph-mon[34495]: pgmap v323: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:04:10.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:09 smithi089 ceph-mon[34495]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T06:04:10.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:09 smithi104 ceph-mon[34559]: pgmap v323: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-05-08T06:04:10.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:09 smithi104 ceph-mon[34559]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T06:04:12.003 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:11 smithi089 ceph-mon[34495]: pgmap v325: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:12.036 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.001106589, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000156306, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:12.037 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 6, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 182, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3264, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 189296, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:12.038 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.016437253, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008736197, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.005229106, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 180, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.017762155, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000098678 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:12.039 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 189296 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 247, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 236, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1717600, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 145344, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:12.040 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.050528480, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.015198789, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.038701728, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.006116794, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 235, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.013955615, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000059385 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:12.041 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1717600, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 145344 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:12.042 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:12.043 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:12.044 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148721, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:12.045 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:12.046 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 249, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1682731, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 249, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 249, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1682731, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:12.047 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:12.048 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-05-08T06:04:12.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:11 smithi003 ceph-mon[30137]: pgmap v325: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:04:12.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:11 smithi104 ceph-mon[34559]: pgmap v325: 129 pgs: 129 active+clean; 2.5 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 22 op/s 2024-05-08T06:04:12.709 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-05-08T06:04:12.709 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-05-08T06:04:12.709 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-05-08T06:04:12.709 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T06:04:12.710 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T06:04:12.946 INFO:teuthology.orchestra.run.smithi089.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T06:04:14.083 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:13 smithi089 ceph-mon[34495]: pgmap v326: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:04:14.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:13 smithi003 ceph-mon[30137]: pgmap v326: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:04:14.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:13 smithi104 ceph-mon[34559]: pgmap v326: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 254, 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 243, 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 1720128, 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 149520, 2024-05-08T06:04:14.624 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.050633035, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.015556978, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.037878866, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.006130302, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 242, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.012903334, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000053319 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.625 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 1720128, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 149520 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 6, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 187, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 3264, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 190176, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.012800124, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.008259801, 2024-05-08T06:04:14.626 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.002004636, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 185, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.022456855, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000121388 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 190176 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.627 INFO:teuthology.orchestra.run.smithi089.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_total_time": 0.000334735, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_recv_time": 0.000274265, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.628 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "cct": { 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "total_workers": 0, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "unhealthy_workers": 0 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "mempool": { 2024-05-08T06:04:14.629 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:14.630 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:14.631 INFO:teuthology.orchestra.run.smithi089.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_items": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_bytes": 148721, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_anon_items": 30, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_bytes": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_items": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:14.632 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_items": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "pgmap_items": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "mds_co_items": 2, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_1_items": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: "unittest_2_items": 0 2024-05-08T06:04:14.633 INFO:teuthology.orchestra.run.smithi089.stdout: }, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "val": 0, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "max": 104857600, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_started": 0, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get": 256, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_sum": 1684278, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "get_or_fail_success": 256, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "take": 0, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "take_sum": 0, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "put": 256, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "put_sum": 1684278, 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "wait": { 2024-05-08T06:04:14.634 INFO:teuthology.orchestra.run.smithi089.stdout: "avgcount": 0, 2024-05-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "sum": 0.000000000, 2024-05-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: "avgtime": 0.000000000 2024-05-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi089.stdout: } 2024-05-08T06:04:14.635 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-05-08T06:04:15.268 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-08T06:04:15.268 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-08T06:04:15.268 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-08T06:04:15.268 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T06:04:15.269 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T06:04:15.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:15 smithi089 ceph-mon[34495]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T06:04:15.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:15 smithi089 ceph-mon[34495]: pgmap v328: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:15.522 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:15.643 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:15 smithi104 ceph-mon[34559]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T06:04:15.643 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:15 smithi104 ceph-mon[34559]: pgmap v328: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:15.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:15 smithi003 ceph-mon[30137]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T06:04:15.660 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:15 smithi003 ceph-mon[30137]: pgmap v328: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 249, 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 238, 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1718144, 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 146624, 2024-05-08T06:04:17.437 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.050466691, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.015459883, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.037909738, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.005966140, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 237, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.011843393, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000049972 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.438 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1718144, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 146624 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 6, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 184, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 3264, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 189648, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:17.439 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.013091083, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.008579175, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.002118846, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 182, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018115595, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000099536 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:17.440 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 189648 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 0, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 2, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 0, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 0, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.000279682, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000225565, 2024-05-08T06:04:17.441 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.442 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:17.443 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:17.444 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148731, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 31, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:17.445 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:17.446 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 251, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1682991, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 251, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-08T06:04:17.447 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 251, 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1682991, 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:17.448 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-08T06:04:17.877 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:17 smithi104 ceph-mon[34559]: pgmap v329: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:04:18.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:17 smithi089 ceph-mon[34495]: pgmap v329: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:04:18.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:17 smithi003 ceph-mon[30137]: pgmap v329: 129 pgs: 129 active+clean; 2.5 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-05-08T06:04:18.237 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-05-08T06:04:18.237 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-05-08T06:04:18.237 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-05-08T06:04:18.237 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T06:04:18.238 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T06:04:18.485 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:20.006 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:19 smithi104 ceph-mon[34559]: pgmap v330: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-05-08T06:04:20.012 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:19 smithi089 ceph-mon[34495]: pgmap v330: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-05-08T06:04:20.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:19 smithi003 ceph-mon[30137]: pgmap v330: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 206 B/s wr, 2 op/s 2024-05-08T06:04:20.258 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-08T06:04:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:20.258 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 4, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 4, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2944, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 448, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.001324200, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000030094, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000512665, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000650009, 2024-05-08T06:04:20.259 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000216669 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 250, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 239, 2024-05-08T06:04:20.260 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1718416, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 147264, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.053483728, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.016280461, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.039809679, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.006507335, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 238, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.014206563, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000059691 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:20.261 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1718416, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 147264 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 183, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 189376, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.013412662, 2024-05-08T06:04:20.262 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.009228833, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.001489680, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 180, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018710565, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000103947 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 189376 2024-05-08T06:04:20.263 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:20.264 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:20.265 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:20.266 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148771, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 32, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:20.267 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 252, 2024-05-08T06:04:20.268 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1683121, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 252, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 252, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1683121, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:20.269 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-08T06:04:20.943 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-05-08T06:04:20.943 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-05-08T06:04:20.943 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-05-08T06:04:20.943 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T06:04:20.943 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T06:04:21.188 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:22.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:21 smithi104 ceph-mon[34559]: pgmap v331: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:04:22.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:21 smithi003 ceph-mon[30137]: pgmap v331: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:04:22.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:21 smithi089 ceph-mon[34495]: pgmap v331: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1363, 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 773, 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 486324, 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 1668749, 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 10, 2024-05-08T06:04:22.891 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.148672886, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.036489467, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.048312129, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.056116483, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 772, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.130637146, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000169219 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 912, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000207629, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000227 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 333296, 2024-05-08T06:04:22.892 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 100480 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 483, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 366, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 159990, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 759094, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 48, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.056369432, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.012250842, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.024005912, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.015400748, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 363, 2024-05-08T06:04:22.893 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.059400015, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000163636 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 257, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000041708, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000162 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 88048, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 177248 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1462, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 932, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2182525, 2024-05-08T06:04:22.894 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 779682, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 15, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.193681924, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.050175137, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.096195937, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.058331617, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 931, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.064560231, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000069345 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 427, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000168352, 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000394 2024-05-08T06:04:22.895 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1822048, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 509696 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 666, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.142469942, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000213918 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.896 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 77, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.003411170, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000044300 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 197, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 197, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 197, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.155890754, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000791323 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.897 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 67, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 42, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 13, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 29, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 23, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 3, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 4, 2024-05-08T06:04:22.898 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 2, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 254, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 246, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-05-08T06:04:22.899 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 6, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 27, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 37, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T06:04:22.900 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 119, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 88, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 3, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 28 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 2, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 6, 2024-05-08T06:04:22.901 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 4, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 2, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 24, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-05-08T06:04:22.902 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 284, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 208, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 208, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 76, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 40, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 15, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 14, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 14, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 1, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 1, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 1, 2024-05-08T06:04:22.903 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 5412460, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 5517644, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 240, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.543607364, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002265030 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 29, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 33, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 4, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 20, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 20, 2024-05-08T06:04:22.904 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 0, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 42, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 43, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 1, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 4, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 27, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 23, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 56684, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 231800 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 226, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 197, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 37, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-05-08T06:04:22.905 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 8, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.006026429, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000753303 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 126, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.024716820, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000196165 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.906 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001764876, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000352975 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.907 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000485507, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000242753 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.908 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001800882, 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000163716 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.909 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 8, 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.011461737, 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001432717 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-05-08T06:04:22.910 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 11, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.046952861, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004268441 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000213832, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000213832 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001214337, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000607168 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-05-08T06:04:22.911 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.002205932, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001102966 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.011344638, 2024-05-08T06:04:22.912 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005672319 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 5, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018222461, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003644492 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.001086897, 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000543448 2024-05-08T06:04:22.913 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.914 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 8, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.021852156, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002731519 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 4, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.006541389, 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001635347 2024-05-08T06:04:22.915 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 2, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 6, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 4, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 2, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 27, 2024-05-08T06:04:22.916 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 13, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 626, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:22.917 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:22.918 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 319683, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 109, 2024-05-08T06:04:22.919 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 9512, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 182, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 140053, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 1179, 2024-05-08T06:04:22.920 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 1048, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 1464778, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 37, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 1048, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1048, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 21.727751745, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.020732587 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.921 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1048, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1256 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 1048, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 2, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 1046, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 47, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 42, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 2, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 242, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 66, 2024-05-08T06:04:22.922 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 583, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 76, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-05-08T06:04:22.923 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-05-08T06:04:22.924 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 101, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 23, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 181, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 0, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 17 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.925 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "osd-slow-ops": { 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 26 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 65, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 3, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 71, 2024-05-08T06:04:22.926 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 4, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 3304, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 2474736, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 3304, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 3304, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 2474736, 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:22.927 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 1048, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 1465246, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1048, 2024-05-08T06:04:22.928 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1465246, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 1048, 2024-05-08T06:04:22.929 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 1048, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1048, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1048, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 4, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 468, 2024-05-08T06:04:22.930 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 4, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 2, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 468, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x560d7ad04460": { 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 284, 2024-05-08T06:04:22.931 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1323340, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 284, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 240, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1323340, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:22.932 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-08T06:04:23.725 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-08T06:04:23.725 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2024-05-08T06:04:23.725 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=5 2024-05-08T06:04:23.725 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T06:04:23.726 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T06:04:23.966 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T06:04:24.020 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:23 smithi104 ceph-mon[34559]: pgmap v332: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:04:24.064 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:23 smithi089 ceph-mon[34495]: pgmap v332: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:04:24.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:23 smithi003 ceph-mon[30137]: pgmap v332: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-05-08T06:04:25.753 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 4, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 4, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 2944, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 448, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 4, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 0, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.001239936, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.000027574, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.000479859, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3, 2024-05-08T06:04:25.754 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000700844, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000233614 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 2, 2024-05-08T06:04:25.755 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 188, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 320, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 190256, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 5, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.013275868, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.009270121, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.001339559, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 185, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.018609998, 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000100594 2024-05-08T06:04:25.756 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 190256 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 257, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 246, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 1720944, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 151440, 2024-05-08T06:04:25.757 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 2, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 1, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.052291900, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.015602664, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.040453504, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.006077217, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 245, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.014607685, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000059623 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:25.758 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1720944, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 151440 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 0, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-05-08T06:04:25.759 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-05-08T06:04:25.760 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-05-08T06:04:25.761 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 148704, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 29, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-05-08T06:04:25.762 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 928, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 2, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-05-08T06:04:25.763 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 259, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 1684668, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 259, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 259, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 1684668, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-05-08T06:04:25.764 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-05-08T06:04:25.765 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-05-08T06:04:25.765 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:25.765 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-05-08T06:04:25.765 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-05-08T06:04:26.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:25 smithi104 ceph-mon[34559]: pgmap v333: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:04:26.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:25 smithi003 ceph-mon[30137]: pgmap v333: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:04:26.258 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:25 smithi089 ceph-mon[34495]: pgmap v333: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 99 B/s wr, 2 op/s 2024-05-08T06:04:26.481 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-08T06:04:26.482 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-08T06:04:26.482 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-08T06:04:26.482 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T06:04:26.493 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T06:04:26.493 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T06:04:26.493 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:04:26.515 DEBUG:teuthology.orchestra.run.smithi003:> set -ex 2024-05-08T06:04:26.515 DEBUG:teuthology.orchestra.run.smithi003:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T06:04:26.576 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T06:04:26.878 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:27.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:27 smithi003 ceph-mon[30137]: pgmap v334: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:04:28.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:27 smithi089 ceph-mon[34495]: pgmap v334: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:04:28.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:27 smithi104 ceph-mon[34559]: pgmap v334: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-05-08T06:04:28.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 -- 172.21.15.3:0/1054159827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 msgr2=0x7fec041103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:28.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 --2- 172.21.15.3:0/1054159827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec041103f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b0d0 tx=0x7febfc00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 -- 172.21.15.3:0/1054159827 shutdown_connections 2024-05-08T06:04:28.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 --2- 172.21.15.3:0/1054159827 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec041103f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.585 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 --2- 172.21.15.3:0/1054159827 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec04103780 0x7fec04103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 --2- 172.21.15.3:0/1054159827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec04104130 0x7fec04108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 -- 172.21.15.3:0/1054159827 >> 172.21.15.3:0/1054159827 conn(0x7fec0407a610 msgr2=0x7fec0407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 -- 172.21.15.3:0/1054159827 shutdown_connections 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.583+0000 7fec08d4e700 1 -- 172.21.15.3:0/1054159827 wait complete. 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 Processor -- start 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 -- start start 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec04103780 0x7fec0419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec04104130 0x7fec0419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec04199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fec04113640 con 0x7fec04108870 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fec041134c0 con 0x7fec04103780 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec08d4e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fec041137c0 con 0x7fec04104130 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec02d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec04199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:28.586 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec02d9d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec04199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44022/0 (socket says 172.21.15.3:44022) 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec02d9d700 1 -- 172.21.15.3:0/363659755 learned_addr learned my addr 172.21.15.3:0/363659755 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec0259c700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec04103780 0x7fec0419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.584+0000 7fec01d9b700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec04104130 0x7fec0419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec02d9d700 1 -- 172.21.15.3:0/363659755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec04104130 msgr2=0x7fec0419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec02d9d700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec04104130 0x7fec0419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec02d9d700 1 -- 172.21.15.3:0/363659755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec04103780 msgr2=0x7fec0419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec02d9d700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec04103780 0x7fec0419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec02d9d700 1 -- 172.21.15.3:0/363659755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec04199fe0 con 0x7fec04108870 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec02d9d700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec04199880 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7febfc0095c0 tx=0x7febfc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc01f070 con 0x7fec04108870 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0419a270 con 0x7fec04108870 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc003c00 con 0x7fec04108870 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc01b6a0 con 0x7fec04108870 2024-05-08T06:04:28.587 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.585+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a6590 con 0x7fec04108870 2024-05-08T06:04:28.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.586+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec04104c00 con 0x7fec04108870 2024-05-08T06:04:28.588 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.587+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7febfc022070 con 0x7fec04108870 2024-05-08T06:04:28.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.588+0000 7febf37fe700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7febec0b29f0 0x7febec0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:28.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.588+0000 7fec0259c700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7febec0b29f0 0x7febec0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:28.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.588+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7febfc00f020 con 0x7fec04108870 2024-05-08T06:04:28.589 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.589+0000 7fec0259c700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7febec0b29f0 0x7febec0b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febf4005950 tx=0x7febf400a300 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:28.591 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.592+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febfc029020 con 0x7fec04108870 2024-05-08T06:04:28.834 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.834+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fec0402e510 con 0x7fec04108870 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.835+0000 7febf37fe700 1 -- 172.21.15.3:0/363659755 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v101) ==== 81+0+926 (secure 0 0 0) 0x7febfc0ab6e0 con 0x7fec04108870 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/2793699832 2024-05-09T05:53:46.504286+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/864931570 2024-05-09T05:53:46.504286+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3161157856 2024-05-09T05:53:46.504286+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/2793699832 2024-05-09T05:53:46.504286+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1128678241 2024-05-09T05:53:46.504286+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3921401880 2024-05-09T05:53:46.504286+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/2743706157 2024-05-09T05:42:33.122111+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/174548513 2024-05-09T05:42:33.122111+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/174548513 2024-05-09T05:42:33.122111+0000 2024-05-08T06:04:28.835 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6800/3412581798 2024-05-09T05:42:06.081817+0000 2024-05-08T06:04:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/615172825 2024-05-09T05:42:33.122111+0000 2024-05-08T06:04:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/4027937918 2024-05-09T05:42:06.081817+0000 2024-05-08T06:04:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:6801/3412581798 2024-05-09T05:42:06.081817+0000 2024-05-08T06:04:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/1221768244 2024-05-09T05:42:06.081817+0000 2024-05-08T06:04:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3389888238 2024-05-09T05:42:06.081817+0000 2024-05-08T06:04:28.836 INFO:teuthology.orchestra.run.smithi003.stdout:172.21.15.3:0/3812286739 2024-05-09T05:42:33.122111+0000 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7febec0b29f0 msgr2=0x7febec0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7febec0b29f0 0x7febec0b4eb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7febf4005950 tx=0x7febf400a300 comp rx=0 tx=0).stop 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 msgr2=0x7fec04199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec04199880 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7febfc0095c0 tx=0x7febfc007a20 comp rx=0 tx=0).stop 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 shutdown_connections 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fec04108870 0x7fec04199880 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fec04103780 0x7fec0419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7febec0b29f0 0x7febec0b4eb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 --2- 172.21.15.3:0/363659755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fec04104130 0x7fec0419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 >> 172.21.15.3:0/363659755 conn(0x7fec0407a610 msgr2=0x7fec04102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 shutdown_connections 2024-05-08T06:04:28.838 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:28.837+0000 7fec08d4e700 1 -- 172.21.15.3:0/363659755 wait complete. 2024-05-08T06:04:28.839 INFO:teuthology.orchestra.run.smithi003.stderr:listed 16 entries 2024-05-08T06:04:29.536 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T06:04:29.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:29.537 DEBUG:teuthology.orchestra.run.smithi003:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T06:04:29.582 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T06:04:29.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:29.582 DEBUG:teuthology.orchestra.run.smithi003:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T06:04:29.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:29.609 DEBUG:teuthology.orchestra.run.smithi003:> ip netns list 2024-05-08T06:04:29.666 INFO:teuthology.orchestra.run.smithi003.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T06:04:29.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:29.666 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T06:04:29.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T06:04:29.734 DEBUG:teuthology.orchestra.run.smithi003:> sudo ip link delete ceph-brx 2024-05-08T06:04:29.838 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T06:04:29.848 INFO:tasks.cephadm:Teardown begin 2024-05-08T06:04:29.849 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T06:04:29.849 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:30.065 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:29 smithi104 ceph-mon[34559]: pgmap v335: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:04:30.065 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:29 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/363659755' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:04:30.105 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:30.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:29 smithi003 ceph-mon[30137]: pgmap v335: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:04:30.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:29 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/363659755' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:04:30.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:29 smithi089 ceph-mon[34495]: pgmap v335: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-05-08T06:04:30.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:29 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/363659755' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T06:04:31.841 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:31 smithi003 ceph-mon[30137]: pgmap v336: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:04:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/3097318664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48104100 msgr2=0x7f7c48108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/3097318664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48104100 0x7f7c48108120 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7c340099c0 tx=0x7f7c34009cd0 comp rx=0 tx=0).stop 2024-05-08T06:04:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/3097318664 shutdown_connections 2024-05-08T06:04:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/3097318664 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c48108850 0x7f7c481103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/3097318664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c48103750 0x7f7c48103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:31.955 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/3097318664 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48104100 0x7f7c48108120 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:31.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/3097318664 >> 172.21.15.3:0/3097318664 conn(0x7f7c4807a600 msgr2=0x7f7c4807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:31.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/3097318664 shutdown_connections 2024-05-08T06:04:31.956 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.955+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/3097318664 wait complete. 2024-05-08T06:04:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 Processor -- start 2024-05-08T06:04:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 -- start start 2024-05-08T06:04:31.957 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 0x7f7c480757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c48104100 0x7f7c48075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c48108850 0x7f7c48076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48113680 con 0x7f7c48103750 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48113500 con 0x7f7c48104100 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ec7d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48113800 con 0x7f7c48108850 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 0x7f7c480757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 0x7f7c480757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:44046/0 (socket says 172.21.15.3:44046) 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 -- 172.21.15.3:0/16250622 learned_addr learned my addr 172.21.15.3:0/16250622 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4d21a700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c48108850 0x7f7c48076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c47fff700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c48104100 0x7f7c48075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 -- 172.21.15.3:0/16250622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c48108850 msgr2=0x7f7c48076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:31.958 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c48108850 0x7f7c48076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:31.959 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 -- 172.21.15.3:0/16250622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c48104100 msgr2=0x7f7c48075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.956+0000 7f7c4ca19700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c48104100 0x7f7c48075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c4ca19700 1 -- 172.21.15.3:0/16250622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c481a80a0 con 0x7f7c48103750 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c4ca19700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 0x7f7c480757a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9c0 tx=0x7f7c3c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c00cbd0 con 0x7f7c48103750 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c3c00cd70 con 0x7f7c48103750 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3c005610 con 0x7f7c48103750 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c481a8390 con 0x7f7c48103750 2024-05-08T06:04:31.960 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.957+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481a8850 con 0x7f7c48103750 2024-05-08T06:04:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.958+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4810ba20 con 0x7f7c48103750 2024-05-08T06:04:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.959+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f7c3c004750 con 0x7f7c48103750 2024-05-08T06:04:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.960+0000 7f7c45ffb700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7c380b29f0 0x7f7c380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.960+0000 7f7c47fff700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7c380b29f0 0x7f7c380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:31.961 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.960+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f7c3c0e2d10 con 0x7f7c48103750 2024-05-08T06:04:31.962 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.961+0000 7f7c47fff700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7c380b29f0 0x7f7c380b4eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c48077230 tx=0x7f7c34005fb0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:31.965 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:31.965+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c3c0aa320 con 0x7f7c48103750 2024-05-08T06:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.234+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c4806a190 con 0x7f7c48103750 2024-05-08T06:04:32.234 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.234+0000 7f7c45ffb700 1 -- 172.21.15.3:0/16250622 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7f7c3c0a9c20 con 0x7f7c48103750 2024-05-08T06:04:32.235 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T06:04:32.235 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7c380b29f0 msgr2=0x7f7c380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7c380b29f0 0x7f7c380b4eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c48077230 tx=0x7f7c34005fb0 comp rx=0 tx=0).stop 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 msgr2=0x7f7c480757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 0x7f7c480757a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c00e9c0 tx=0x7f7c3c00ecd0 comp rx=0 tx=0).stop 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 shutdown_connections 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f7c380b29f0 0x7f7c380b4eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:32.237 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c48108850 0x7f7c48076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f7c48103750 0x7f7c480757a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 --2- 172.21.15.3:0/16250622 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7c48104100 0x7f7c48075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 >> 172.21.15.3:0/16250622 conn(0x7f7c4807a600 msgr2=0x7f7c48101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 shutdown_connections 2024-05-08T06:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:32.237+0000 7f7c4ec7d700 1 -- 172.21.15.3:0/16250622 wait complete. 2024-05-08T06:04:32.238 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T06:04:32.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:31 smithi089 ceph-mon[34495]: pgmap v336: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:04:32.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:31 smithi104 ceph-mon[34559]: pgmap v336: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-08T06:04:32.844 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:32 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/16250622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:32.873 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T06:04:33.124 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:33.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:32 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/16250622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:33.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:32 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/16250622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:33.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:33 smithi003 ceph-mon[30137]: pgmap v337: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:34.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:33 smithi089 ceph-mon[34495]: pgmap v337: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:34.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:33 smithi104 ceph-mon[34559]: pgmap v337: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:34.923 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 -- 172.21.15.3:0/1067912310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 msgr2=0x7f901c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 --2- 172.21.15.3:0/1067912310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c102a70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f900c00b0d0 tx=0x7f900c00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 -- 172.21.15.3:0/1067912310 shutdown_connections 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 --2- 172.21.15.3:0/1067912310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f901c1076e0 0x7f901c10f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 --2- 172.21.15.3:0/1067912310 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c102a70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 --2- 172.21.15.3:0/1067912310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f901c103040 0x7f901c107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 -- 172.21.15.3:0/1067912310 >> 172.21.15.3:0/1067912310 conn(0x7f901c0783f0 msgr2=0x7f901c078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 -- 172.21.15.3:0/1067912310 shutdown_connections 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.923+0000 7f90220e2700 1 -- 172.21.15.3:0/1067912310 wait complete. 2024-05-08T06:04:34.924 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 Processor -- start 2024-05-08T06:04:34.926 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 -- start start 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f901c103040 0x7f901c19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f901c1076e0 0x7f901c199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f901c1125a0 con 0x7f901c102690 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f901c112420 con 0x7f901c103040 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f90220e2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f901c112720 con 0x7f901c1076e0 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f901b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f901b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38148/0 (socket says 172.21.15.3:38148) 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.924+0000 7f901b7fe700 1 -- 172.21.15.3:0/3307039510 learned_addr learned my addr 172.21.15.3:0/3307039510 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901bfff700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f901c1076e0 0x7f901c199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:34.927 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901affd700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f901c103040 0x7f901c19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901b7fe700 1 -- 172.21.15.3:0/3307039510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f901c1076e0 msgr2=0x7f901c199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901b7fe700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f901c1076e0 0x7f901c199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901b7fe700 1 -- 172.21.15.3:0/3307039510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f901c103040 msgr2=0x7f901c19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901b7fe700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f901c103040 0x7f901c19fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901b7fe700 1 -- 172.21.15.3:0/3307039510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901c19a180 con 0x7f901c102690 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f901b7fe700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c19f810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f900c00d010 tx=0x7f900c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c01f070 con 0x7f901c102690 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f900c003c00 con 0x7f901c102690 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901c19a410 con 0x7f901c102690 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c01b6a0 con 0x7f901c102690 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.925+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f901c1a66e0 con 0x7f901c102690 2024-05-08T06:04:34.928 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.926+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901c06a190 con 0x7f901c102690 2024-05-08T06:04:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.928+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f900c022070 con 0x7f901c102690 2024-05-08T06:04:34.929 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.929+0000 7f9018ff9700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f90080b29f0 0x7f90080b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.929+0000 7f901affd700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f90080b29f0 0x7f90080b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.929+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f900c00f020 con 0x7f901c102690 2024-05-08T06:04:34.930 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.929+0000 7f901affd700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f90080b29f0 0x7f90080b4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9004005fd0 tx=0x7f900400a560 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:34.932 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:34.932+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f900c029020 con 0x7f901c102690 2024-05-08T06:04:35.201 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.201+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f901c02e510 con 0x7f901c102690 2024-05-08T06:04:35.202 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.202+0000 7f9018ff9700 1 -- 172.21.15.3:0/3307039510 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11800 (secure 0 0 0) 0x7f900c0ab730 con 0x7f901c102690 2024-05-08T06:04:35.202 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T06:04:35.202 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14499,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6835/2960751422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6834","nonce":2960751422},{"type":"v1","addr":"172.21.15.3:6835","nonce":2960751422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14511,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6837/2285392354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6836","nonce":2285392354},{"type":"v1","addr":"172.21.15.3:6837","nonce":2285392354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14559,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6839/3875310227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6838","nonce":3875310227},{"type":"v1","addr":"172.21.15.3:6839","nonce":3875310227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.3:6841/1028446466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6840","nonce":1028446466},{"type":"v1","addr":"172.21.15.3:6841","nonce":1028446466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6833/3884583981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6832","nonce":3884583981},{"type":"v1","addr":"172.21.15.89:6833","nonce":3884583981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6835/3820045271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6834","nonce":3820045271},{"type":"v1","addr":"172.21.15.89:6835","nonce":3820045271}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/1254707053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":1254707053},{"type":"v1","addr":"172.21.15.104:6833","nonce":1254707053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.89:6837/1710942152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6836","nonce":1710942152},{"type":"v1","addr":"172.21.15.89:6837","nonce":1710942152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/3213984653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":3213984653},{"type":"v1","addr":"172.21.15.104:6835","nonce":3213984653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24481,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2023592505","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2023592505},{"type":"v1","addr":"172.21.15.104:6837","nonce":2023592505}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:50:01.809365+0000","modified":"2024-05-08T05:54:15.627053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24493":{"gid":24493,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6839/3320987620","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":3320987620},{"type":"v1","addr":"172.21.15.104:6839","nonce":3320987620}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25040":{"gid":25040,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.89:6839/3215381485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6838","nonce":3215381485},{"type":"v1","addr":"172.21.15.89:6839","nonce":3215381485}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24493,"qdb_cluster":[24493]},"id":1}]} 2024-05-08T06:04:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f90080b29f0 msgr2=0x7f90080b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f90080b29f0 0x7f90080b4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9004005fd0 tx=0x7f900400a560 comp rx=0 tx=0).stop 2024-05-08T06:04:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 msgr2=0x7f901c19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:35.204 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c19f810 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f900c00d010 tx=0x7f900c007ad0 comp rx=0 tx=0).stop 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 shutdown_connections 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f90080b29f0 0x7f90080b4eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f901c1076e0 0x7f901c199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f901c102690 0x7f901c19f810 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 --2- 172.21.15.3:0/3307039510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f901c103040 0x7f901c19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 >> 172.21.15.3:0/3307039510 conn(0x7f901c0783f0 msgr2=0x7f901c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 shutdown_connections 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:35.204+0000 7f90220e2700 1 -- 172.21.15.3:0/3307039510 wait complete. 2024-05-08T06:04:35.205 INFO:teuthology.orchestra.run.smithi003.stderr:dumped fsmap epoch 23 2024-05-08T06:04:35.885 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:35 smithi003 ceph-mon[30137]: pgmap v338: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:35.885 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:35 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3307039510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:35.918 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T06:04:36.174 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:36.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:35 smithi089 ceph-mon[34495]: pgmap v338: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:36.302 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:35 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3307039510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:36.303 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:35 smithi104 ceph-mon[34559]: pgmap v338: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:36.303 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:35 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3307039510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.823+0000 7f3278e87700 1 -- 172.21.15.3:0/1573378326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 msgr2=0x7f3274110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.823+0000 7f3278e87700 1 --2- 172.21.15.3:0/1573378326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f3274110330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f326c00b0a0 tx=0x7f326c00b3b0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 -- 172.21.15.3:0/1573378326 shutdown_connections 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 --2- 172.21.15.3:0/1573378326 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f3274110330 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 --2- 172.21.15.3:0/1573378326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3274109510 0x7f32741098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 --2- 172.21.15.3:0/1573378326 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f327406b180 0x7f327406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.826 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 -- 172.21.15.3:0/1573378326 >> 172.21.15.3:0/1573378326 conn(0x7f3274079350 msgr2=0x7f3274079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 -- 172.21.15.3:0/1573378326 shutdown_connections 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 -- 172.21.15.3:0/1573378326 wait complete. 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 Processor -- start 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.824+0000 7f3278e87700 1 -- start start 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3278e87700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f327406b180 0x7f327419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3278e87700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f327419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3278e87700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274109510 0x7f32741999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3278e87700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f3274113640 con 0x7f327406bb40 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3278e87700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f32741134c0 con 0x7f3274109510 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3278e87700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f32741137c0 con 0x7f327406b180 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f327419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f327419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38162/0 (socket says 172.21.15.3:38162) 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 -- 172.21.15.3:0/3093882661 learned_addr learned my addr 172.21.15.3:0/3093882661 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:37.827 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f327259c700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f327406b180 0x7f327419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3272d9d700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274109510 0x7f32741999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 -- 172.21.15.3:0/3093882661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f327406b180 msgr2=0x7f327419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f327406b180 0x7f327419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 -- 172.21.15.3:0/3093882661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274109510 msgr2=0x7f32741999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274109510 0x7f32741999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 -- 172.21.15.3:0/3093882661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327419a100 con 0x7f327406bb40 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.825+0000 7f3271d9b700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f327419fd20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f326800d950 tx=0x7f326800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.826+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268016070 con 0x7f327406bb40 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.826+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3268009df0 con 0x7f327406bb40 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.826+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327419a3f0 con 0x7f327406bb40 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.826+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326800f3c0 con 0x7f327406bb40 2024-05-08T06:04:37.828 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.826+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32741a66b0 con 0x7f327406bb40 2024-05-08T06:04:37.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.827+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327410b9f0 con 0x7f327406bb40 2024-05-08T06:04:37.829 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.828+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f3268019070 con 0x7f327406bb40 2024-05-08T06:04:37.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.829+0000 7f32637fe700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f325c0b2a70 0x7f325c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:37.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.829+0000 7f327259c700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f325c0b2a70 0x7f325c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:37.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.829+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7f32680e2520 con 0x7f327406bb40 2024-05-08T06:04:37.830 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.830+0000 7f327259c700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f325c0b2a70 0x7f325c0b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f327419aae0 tx=0x7f3264009260 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:37.832 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:37.833+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f32680a9b70 con 0x7f327406bb40 2024-05-08T06:04:37.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:37 smithi003 ceph-mon[30137]: pgmap v339: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:38.051 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.051+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f327419aed0 con 0x7f327406bb40 2024-05-08T06:04:38.052 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.052+0000 7f32637fe700 1 -- 172.21.15.3:0/3093882661 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25223 (secure 0 0 0) 0x7f32680a9470 con 0x7f327406bb40 2024-05-08T06:04:38.053 INFO:teuthology.orchestra.run.smithi003.stdout: 2024-05-08T06:04:38.053 INFO:teuthology.orchestra.run.smithi003.stdout:{"epoch":101,"fsid":"708b9316-0cfd-11ef-bc97-c7b262605968","created":"2024-05-08T05:41:33.921408+0000","modified":"2024-05-08T06:04:14.395379+0000","last_up_change":"2024-05-08T05:48:54.561587+0000","last_in_change":"2024-05-08T05:48:34.396176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:45:01.214856+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:49:54.940678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:49:58.453825+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:50:10.686434+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"8a64e1a5-0f5b-4a16-b521-3b41c78d7269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6802","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6803","nonce":3730465717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6804","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6805","nonce":3730465717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6808","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6809","nonce":3730465717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6806","nonce":3730465717},{"type":"v1","addr":"172.21.15.3:6807","nonce":3730465717}]},"public_addr":"172.21.15.3:6803/3730465717","cluster_addr":"172.21.15.3:6805/3730465717","heartbeat_back_addr":"172.21.15.3:6809/3730465717","heartbeat_front_addr":"172.21.15.3:6807/3730465717","state":["exists","up"]},{"osd":1,"uuid":"804db558-09e6-4d7d-9eca-b70cd45c71a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6801","nonce":662806549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6803","nonce":662806549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6807","nonce":662806549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":662806549},{"type":"v1","addr":"172.21.15.89:6805","nonce":662806549}]},"public_addr":"172.21.15.89:6801/662806549","cluster_addr":"172.21.15.89:6803/662806549","heartbeat_back_addr":"172.21.15.89:6807/662806549","heartbeat_front_addr":"172.21.15.89:6805/662806549","state":["exists","up"]},{"osd":2,"uuid":"c61371f5-cc2f-45a6-88b2-eefe6596c9b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6801","nonce":3444157225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6803","nonce":3444157225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6807","nonce":3444157225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3444157225},{"type":"v1","addr":"172.21.15.104:6805","nonce":3444157225}]},"public_addr":"172.21.15.104:6801/3444157225","cluster_addr":"172.21.15.104:6803/3444157225","heartbeat_back_addr":"172.21.15.104:6807/3444157225","heartbeat_front_addr":"172.21.15.104:6805/3444157225","state":["exists","up"]},{"osd":3,"uuid":"08946cb5-3b97-40ee-a729-6c9a386aeeb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6810","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6811","nonce":2348325148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6812","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6813","nonce":2348325148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6816","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6817","nonce":2348325148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6814","nonce":2348325148},{"type":"v1","addr":"172.21.15.3:6815","nonce":2348325148}]},"public_addr":"172.21.15.3:6811/2348325148","cluster_addr":"172.21.15.3:6813/2348325148","heartbeat_back_addr":"172.21.15.3:6817/2348325148","heartbeat_front_addr":"172.21.15.3:6815/2348325148","state":["exists","up"]},{"osd":4,"uuid":"c7f59b73-f06d-4487-ad40-019eb1a6b470","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6808","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6809","nonce":3658777632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6810","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6811","nonce":3658777632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6814","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6815","nonce":3658777632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6812","nonce":3658777632},{"type":"v1","addr":"172.21.15.89:6813","nonce":3658777632}]},"public_addr":"172.21.15.89:6809/3658777632","cluster_addr":"172.21.15.89:6811/3658777632","heartbeat_back_addr":"172.21.15.89:6815/3658777632","heartbeat_front_addr":"172.21.15.89:6813/3658777632","state":["exists","up"]},{"osd":5,"uuid":"5d8e47b2-6008-416b-ba61-f08cf12933b9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6809","nonce":4290182422}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6811","nonce":4290182422}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6815","nonce":4290182422}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4290182422},{"type":"v1","addr":"172.21.15.104:6813","nonce":4290182422}]},"public_addr":"172.21.15.104:6809/4290182422","cluster_addr":"172.21.15.104:6811/4290182422","heartbeat_back_addr":"172.21.15.104:6815/4290182422","heartbeat_front_addr":"172.21.15.104:6813/4290182422","state":["exists","up"]},{"osd":6,"uuid":"7ace0899-0a39-4753-a710-047f463b5a7f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6818","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6819","nonce":1882383850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6820","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6821","nonce":1882383850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6824","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6825","nonce":1882383850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6822","nonce":1882383850},{"type":"v1","addr":"172.21.15.3:6823","nonce":1882383850}]},"public_addr":"172.21.15.3:6819/1882383850","cluster_addr":"172.21.15.3:6821/1882383850","heartbeat_back_addr":"172.21.15.3:6825/1882383850","heartbeat_front_addr":"172.21.15.3:6823/1882383850","state":["exists","up"]},{"osd":7,"uuid":"13aa9f3f-8b0f-43e4-b328-653822cc6b28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6816","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6817","nonce":2376892451}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6818","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6819","nonce":2376892451}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6822","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6823","nonce":2376892451}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6820","nonce":2376892451},{"type":"v1","addr":"172.21.15.89:6821","nonce":2376892451}]},"public_addr":"172.21.15.89:6817/2376892451","cluster_addr":"172.21.15.89:6819/2376892451","heartbeat_back_addr":"172.21.15.89:6823/2376892451","heartbeat_front_addr":"172.21.15.89:6821/2376892451","state":["exists","up"]},{"osd":8,"uuid":"a4924e7d-921a-4cbc-a59a-4b23e570e1c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6817","nonce":1691421157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6819","nonce":1691421157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6823","nonce":1691421157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1691421157},{"type":"v1","addr":"172.21.15.104:6821","nonce":1691421157}]},"public_addr":"172.21.15.104:6817/1691421157","cluster_addr":"172.21.15.104:6819/1691421157","heartbeat_back_addr":"172.21.15.104:6823/1691421157","heartbeat_front_addr":"172.21.15.104:6821/1691421157","state":["exists","up"]},{"osd":9,"uuid":"99b34df5-382a-42ff-a9ed-12c6a02706a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6826","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6827","nonce":1455470831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6828","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6829","nonce":1455470831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6832","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6833","nonce":1455470831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.3:6830","nonce":1455470831},{"type":"v1","addr":"172.21.15.3:6831","nonce":1455470831}]},"public_addr":"172.21.15.3:6827/1455470831","cluster_addr":"172.21.15.3:6829/1455470831","heartbeat_back_addr":"172.21.15.3:6833/1455470831","heartbeat_front_addr":"172.21.15.3:6831/1455470831","state":["exists","up"]},{"osd":10,"uuid":"4e939df5-46fd-4a17-ba2d-e3f72120e35e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6824","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6825","nonce":3735647304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6826","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6827","nonce":3735647304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6830","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6831","nonce":3735647304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6828","nonce":3735647304},{"type":"v1","addr":"172.21.15.89:6829","nonce":3735647304}]},"public_addr":"172.21.15.89:6825/3735647304","cluster_addr":"172.21.15.89:6827/3735647304","heartbeat_back_addr":"172.21.15.89:6831/3735647304","heartbeat_front_addr":"172.21.15.89:6829/3735647304","state":["exists","up"]},{"osd":11,"uuid":"262280c4-ebf6-4ea0-ba58-3c2eeab79323","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6825","nonce":1990686084}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6827","nonce":1990686084}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6831","nonce":1990686084}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":1990686084},{"type":"v1","addr":"172.21.15.104:6829","nonce":1990686084}]},"public_addr":"172.21.15.104:6825/1990686084","cluster_addr":"172.21.15.104:6827/1990686084","heartbeat_back_addr":"172.21.15.104:6831/1990686084","heartbeat_front_addr":"172.21.15.104:6829/1990686084","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:11.206167+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:34.754176+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:44:58.916688+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:24.893837+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:45:49.900523+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:14.610142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:46:42.137545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:07.758579+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:47:32.850152+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:00.855960+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:26.927426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:48:52.873824+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.3:6801/2793699832":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/864931570":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/3161157856":"2024-05-09T05:53:46.504286+0000","172.21.15.3:6800/2793699832":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/1128678241":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/3921401880":"2024-05-09T05:53:46.504286+0000","172.21.15.3:0/2743706157":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6801/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/174548513":"2024-05-09T05:42:33.122111+0000","172.21.15.3:6800/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/615172825":"2024-05-09T05:42:33.122111+0000","172.21.15.3:0/4027937918":"2024-05-09T05:42:06.081817+0000","172.21.15.3:6801/3412581798":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/1221768244":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3389888238":"2024-05-09T05:42:06.081817+0000","172.21.15.3:0/3812286739":"2024-05-09T05:42:33.122111+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f325c0b2a70 msgr2=0x7f325c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f325c0b2a70 0x7f325c0b4f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f327419aae0 tx=0x7f3264009260 comp rx=0 tx=0).stop 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 msgr2=0x7f327419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f327419fd20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f326800d950 tx=0x7f326800dc60 comp rx=0 tx=0).stop 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 shutdown_connections 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3274109510 0x7f32741999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f325c0b2a70 0x7f325c0b4f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f327406b180 0x7f327419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 --2- 172.21.15.3:0/3093882661 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f327406bb40 0x7f327419fd20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 >> 172.21.15.3:0/3093882661 conn(0x7f3274079350 msgr2=0x7f3274101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 shutdown_connections 2024-05-08T06:04:38.055 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:38.055+0000 7f3278e87700 1 -- 172.21.15.3:0/3093882661 wait complete. 2024-05-08T06:04:38.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:37 smithi089 ceph-mon[34495]: pgmap v339: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:38.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:37 smithi104 ceph-mon[34559]: pgmap v339: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:38.797 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-08T06:04:39.048 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:39.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:38 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3093882661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:39.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:38 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3093882661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:39.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:38 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3093882661' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T06:04:40.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:39 smithi003 ceph-mon[30137]: pgmap v340: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:40.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:39 smithi089 ceph-mon[34495]: pgmap v340: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:40.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:39 smithi104 ceph-mon[34559]: pgmap v340: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 -- 172.21.15.3:0/983416603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 msgr2=0x7fd48c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 --2- 172.21.15.3:0/983416603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c103b30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fd4880099c0 tx=0x7fd488009cd0 comp rx=0 tx=0).stop 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 -- 172.21.15.3:0/983416603 shutdown_connections 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 --2- 172.21.15.3:0/983416603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48c108850 0x7fd48c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 --2- 172.21.15.3:0/983416603 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c103b30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 --2- 172.21.15.3:0/983416603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c104100 0x7fd48c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 -- 172.21.15.3:0/983416603 >> 172.21.15.3:0/983416603 conn(0x7fd48c07a600 msgr2=0x7fd48c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 -- 172.21.15.3:0/983416603 shutdown_connections 2024-05-08T06:04:40.745 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.744+0000 7fd493824700 1 -- 172.21.15.3:0/983416603 wait complete. 2024-05-08T06:04:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 Processor -- start 2024-05-08T06:04:40.746 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 -- start start 2024-05-08T06:04:40.747 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c104100 0x7fd48c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48c108850 0x7fd48c1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1136a0 con 0x7fd48c103750 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113520 con 0x7fd48c108850 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd493824700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113820 con 0x7fd48c104100 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd4915c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd4915c0700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:38182/0 (socket says 172.21.15.3:38182) 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.745+0000 7fd4915c0700 1 -- 172.21.15.3:0/775912011 learned_addr learned my addr 172.21.15.3:0/775912011 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd491dc1700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48c108850 0x7fd48c1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:40.748 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd490dbf700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c104100 0x7fd48c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4915c0700 1 -- 172.21.15.3:0/775912011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c104100 msgr2=0x7fd48c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4915c0700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c104100 0x7fd48c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4915c0700 1 -- 172.21.15.3:0/775912011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48c108850 msgr2=0x7fd48c1998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4915c0700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48c108850 0x7fd48c1998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4915c0700 1 -- 172.21.15.3:0/775912011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19a040 con 0x7fd48c103750 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4915c0700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c19f720 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098e0 tx=0x7fd4880057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488027070 con 0x7fd48c103750 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd488004020 con 0x7fd48c103750 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19a2d0 con 0x7fd48c103750 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488022e10 con 0x7fd48c103750 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.746+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a65f0 con 0x7fd48c103750 2024-05-08T06:04:40.749 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.747+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c104be0 con 0x7fd48c103750 2024-05-08T06:04:40.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.749+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fd48802a070 con 0x7fd48c103750 2024-05-08T06:04:40.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.749+0000 7fd4827fc700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd4780b29f0 0x7fd4780b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:40.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.749+0000 7fd490dbf700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd4780b29f0 0x7fd4780b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:40.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.749+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9496+0+0 (secure 0 0 0) 0x7fd488005030 con 0x7fd48c103750 2024-05-08T06:04:40.750 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.750+0000 7fd490dbf700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd4780b29f0 0x7fd4780b4eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005ea0 tx=0x7fd47c005e30 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:40.753 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:40.753+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd488031020 con 0x7fd48c103750 2024-05-08T06:04:41.017 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.017+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd48c02e510 con 0x7fd48c103750 2024-05-08T06:04:41.019 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.019+0000 7fd4827fc700 1 -- 172.21.15.3:0/775912011 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7fd488013070 con 0x7fd48c103750 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd4780b29f0 msgr2=0x7fd4780b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd4780b29f0 0x7fd4780b4eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd47c005ea0 tx=0x7fd47c005e30 comp rx=0 tx=0).stop 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 msgr2=0x7fd48c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c19f720 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd4880098e0 tx=0x7fd4880057c0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 shutdown_connections 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd48c108850 0x7fd48c1998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.023 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fd48c103750 0x7fd48c19f720 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fd4780b29f0 0x7fd4780b4eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 --2- 172.21.15.3:0/775912011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd48c104100 0x7fd48c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 >> 172.21.15.3:0/775912011 conn(0x7fd48c07a600 msgr2=0x7fd48c102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 shutdown_connections 2024-05-08T06:04:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:41.023+0000 7fd493824700 1 -- 172.21.15.3:0/775912011 wait complete. 2024-05-08T06:04:41.024 INFO:teuthology.orchestra.run.smithi003.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T06:04:41.332 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a[30133]: 2024-05-08T06:04:41.017+0000 7fec8b9d4700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:04:41.512 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[68662]: ignoring --setuser ceph since I am not root 2024-05-08T06:04:41.512 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[68662]: ignoring --setgroup ceph since I am not root 2024-05-08T06:04:41.513 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mds[68666]: -- 172.21.15.89:0/3408058581 <== mon.2 v2:172.21.15.89:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9e7235a40 con 0x55f9e7209000 2024-05-08T06:04:41.513 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[68662]: starting mds.b at 2024-05-08T06:04:41.521 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:04:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: ignoring --setuser ceph since I am not root 2024-05-08T06:04:41.521 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:04:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: ignoring --setgroup ceph since I am not root 2024-05-08T06:04:41.521 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mds[65302]: -- 172.21.15.104:0/3228041610 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559dc63fba40 con 0x559dc63cec00 2024-05-08T06:04:41.521 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:04:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: starting mds.i at 2024-05-08T06:04:41.672 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T06:04:41.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: pgmap v341: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:41.909 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/775912011' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:04:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:04:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:04:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/775912011' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:04:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T06:04:41.910 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:41 smithi003 ceph-mon[30137]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-08T06:04:41.928 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:42.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: pgmap v341: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:42.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/775912011' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:04:42.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:04:42.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:04:42.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/775912011' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:04:42.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T06:04:42.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:41 smithi089 ceph-mon[34495]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: pgmap v341: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/775912011' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/775912011' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T06:04:42.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:41 smithi104 ceph-mon[34559]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-08T06:04:43.371 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:43 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.89:6838/1288489175,v1:172.21.15.89:6839/1288489175] up:boot 2024-05-08T06:04:43.372 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:43 smithi003 ceph-mon[30137]: mds.? [v2:172.21.15.104:6838/1488933396,v1:172.21.15.104:6839/1488933396] up:boot 2024-05-08T06:04:43.372 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:43 smithi003 ceph-mon[30137]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-08T06:04:43.372 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:43 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:04:43.372 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:43 smithi003 ceph-mon[30137]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:04:43.372 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:43 smithi003 ceph-mon[30137]: pgmap v343: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:43.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:43 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.89:6838/1288489175,v1:172.21.15.89:6839/1288489175] up:boot 2024-05-08T06:04:43.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:43 smithi089 ceph-mon[34495]: mds.? [v2:172.21.15.104:6838/1488933396,v1:172.21.15.104:6839/1488933396] up:boot 2024-05-08T06:04:43.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:43 smithi089 ceph-mon[34495]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-08T06:04:43.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:43 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:04:43.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:43 smithi089 ceph-mon[34495]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:04:43.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:43 smithi089 ceph-mon[34495]: pgmap v343: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:43.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:43 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.89:6838/1288489175,v1:172.21.15.89:6839/1288489175] up:boot 2024-05-08T06:04:43.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:43 smithi104 ceph-mon[34559]: mds.? [v2:172.21.15.104:6838/1488933396,v1:172.21.15.104:6839/1488933396] up:boot 2024-05-08T06:04:43.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:43 smithi104 ceph-mon[34559]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-08T06:04:43.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:43 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T06:04:43.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:43 smithi104 ceph-mon[34559]: from='mgr.15129 172.21.15.3:0/2184894182' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T06:04:43.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:43 smithi104 ceph-mon[34559]: pgmap v343: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:43.629 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 -- 172.21.15.3:0/762607296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f01024d0 msgr2=0x7f95f0102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 --2- 172.21.15.3:0/762607296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f01024d0 0x7f95f0102950 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009990 tx=0x7f95d8009ca0 comp rx=0 tx=0).stop 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 -- 172.21.15.3:0/762607296 shutdown_connections 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 --2- 172.21.15.3:0/762607296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95f0102e90 0x7f95f010f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 --2- 172.21.15.3:0/762607296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0108480 0x7f95f0108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 --2- 172.21.15.3:0/762607296 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f01024d0 0x7f95f0102950 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 -- 172.21.15.3:0/762607296 >> 172.21.15.3:0/762607296 conn(0x7f95f00782b0 msgr2=0x7f95f00786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 -- 172.21.15.3:0/762607296 shutdown_connections 2024-05-08T06:04:43.630 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.629+0000 7f95f643b700 1 -- 172.21.15.3:0/762607296 wait complete. 2024-05-08T06:04:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 Processor -- start 2024-05-08T06:04:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 -- start start 2024-05-08T06:04:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95f01024d0 0x7f95f019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:43.632 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0102e90 0x7f95f019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 0x7f95f01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0112590 con 0x7f95f0108480 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0112410 con 0x7f95f0102e90 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.630+0000 7f95f643b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0112710 con 0x7f95f01024d0 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 0x7f95f01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0102e90 0x7f95f019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 0x7f95f01999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40804/0 (socket says 172.21.15.3:40804) 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95e7fff700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0102e90 0x7f95f019fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.3:41856/0 (socket says 172.21.15.3:41856) 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 -- 172.21.15.3:0/3997633343 learned_addr learned my addr 172.21.15.3:0/3997633343 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95effff700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95f01024d0 0x7f95f019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 -- 172.21.15.3:0/3997633343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95f01024d0 msgr2=0x7f95f019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95f01024d0 0x7f95f019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 -- 172.21.15.3:0/3997633343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0102e90 msgr2=0x7f95f019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:43.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0102e90 0x7f95f019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 -- 172.21.15.3:0/3997633343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f019a130 con 0x7f95f0108480 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f49d8700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 0x7f95f01999d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007d70 tx=0x7f95e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8016020 con 0x7f95f0108480 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e800ac90 con 0x7f95f0108480 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f019a420 con 0x7f95f0108480 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.631+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80115e0 con 0x7f95f0108480 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.632+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01a66e0 con 0x7f95f0108480 2024-05-08T06:04:43.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.632+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f006a190 con 0x7f95f0108480 2024-05-08T06:04:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.634+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f95e8019020 con 0x7f95f0108480 2024-05-08T06:04:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.635+0000 7f95edffb700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f95dc0b29f0 0x7f95dc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.635+0000 7f95effff700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f95dc0b29f0 0x7f95dc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.635+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9494+0+0 (secure 0 0 0) 0x7f95e80e2dc0 con 0x7f95f0108480 2024-05-08T06:04:43.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.635+0000 7f95effff700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f95dc0b29f0 0x7f95dc0b4eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f95f019ab10 tx=0x7f95e000b410 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:43.638 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.638+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f95e8020020 con 0x7f95f0108480 2024-05-08T06:04:43.903 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:43.903+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f95f019af00 con 0x7f95f0108480 2024-05-08T06:04:44.028 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.028+0000 7f95edffb700 1 -- 172.21.15.3:0/3997633343 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f95e80aa480 con 0x7f95f0108480 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f95dc0b29f0 msgr2=0x7f95dc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f95dc0b29f0 0x7f95dc0b4eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f95f019ab10 tx=0x7f95e000b410 comp rx=0 tx=0).stop 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 msgr2=0x7f95f01999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 0x7f95f01999d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007d70 tx=0x7f95e800a340 comp rx=0 tx=0).stop 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 shutdown_connections 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f95f0108480 0x7f95f01999d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f95dc0b29f0 0x7f95dc0b4eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95f01024d0 0x7f95f019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 --2- 172.21.15.3:0/3997633343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f95f0102e90 0x7f95f019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.031+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 >> 172.21.15.3:0/3997633343 conn(0x7f95f00782b0 msgr2=0x7f95f006c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:44.032 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.032+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 shutdown_connections 2024-05-08T06:04:44.033 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:44.032+0000 7f95f643b700 1 -- 172.21.15.3:0/3997633343 wait complete. 2024-05-08T06:04:44.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:44 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3997633343' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:04:44.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:44 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3997633343' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:04:44.520 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:44 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3997633343' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T06:04:44.682 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T06:04:44.938 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:45.311 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:45 smithi003 ceph-mon[30137]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:04:45.311 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:45 smithi003 ceph-mon[30137]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:04:45.312 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:45 smithi003 ceph-mon[30137]: Cluster is now healthy 2024-05-08T06:04:45.312 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:45 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3997633343' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:04:45.312 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:45 smithi003 ceph-mon[30137]: fsmap 12 up:standby 2024-05-08T06:04:45.312 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:45 smithi003 ceph-mon[30137]: pgmap v344: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:45.512 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:45 smithi089 ceph-mon[34495]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:04:45.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:45 smithi089 ceph-mon[34495]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:04:45.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:45 smithi089 ceph-mon[34495]: Cluster is now healthy 2024-05-08T06:04:45.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:45 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3997633343' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:04:45.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:45 smithi089 ceph-mon[34495]: fsmap 12 up:standby 2024-05-08T06:04:45.513 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:45 smithi089 ceph-mon[34495]: pgmap v344: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:45 smithi104 ceph-mon[34559]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T06:04:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:45 smithi104 ceph-mon[34559]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T06:04:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:45 smithi104 ceph-mon[34559]: Cluster is now healthy 2024-05-08T06:04:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:45 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3997633343' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T06:04:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:45 smithi104 ceph-mon[34559]: fsmap 12 up:standby 2024-05-08T06:04:45.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:45 smithi104 ceph-mon[34559]: pgmap v344: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 -- 172.21.15.3:0/1667723586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 msgr2=0x7ff24c112290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 --2- 172.21.15.3:0/1667723586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c112290 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff24400b0a0 tx=0x7ff24400b3b0 comp rx=0 tx=0).stop 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 -- 172.21.15.3:0/1667723586 shutdown_connections 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 --2- 172.21.15.3:0/1667723586 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c112290 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 --2- 172.21.15.3:0/1667723586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff24c0756d0 0x7ff24c075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 --2- 172.21.15.3:0/1667723586 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff24c075ff0 0x7ff24c10f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:46.633 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 -- 172.21.15.3:0/1667723586 >> 172.21.15.3:0/1667723586 conn(0x7ff24c07a600 msgr2=0x7ff24c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 -- 172.21.15.3:0/1667723586 shutdown_connections 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.631+0000 7ff253b2b700 1 -- 172.21.15.3:0/1667723586 wait complete. 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 Processor -- start 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 -- start start 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff24c0756d0 0x7ff24c1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff24c075ff0 0x7ff24c1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c115880 con 0x7ff24c10fea0 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c115700 con 0x7ff24c075ff0 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff253b2b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c115a00 con 0x7ff24c0756d0 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:46.634 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40822/0 (socket says 172.21.15.3:40822) 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 -- 172.21.15.3:0/4291003626 learned_addr learned my addr 172.21.15.3:0/4291003626 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2510c6700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff24c075ff0 0x7ff24c1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2518c7700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff24c0756d0 0x7ff24c1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 -- 172.21.15.3:0/4291003626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff24c0756d0 msgr2=0x7ff24c1a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff24c0756d0 0x7ff24c1a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 -- 172.21.15.3:0/4291003626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff24c075ff0 msgr2=0x7ff24c1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff24c075ff0 0x7ff24c1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.632+0000 7ff2520c8700 1 -- 172.21.15.3:0/4291003626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19c240 con 0x7ff24c10fea0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.633+0000 7ff2520c8700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c19bae0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff24400b830 tx=0x7ff244004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.633+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244021070 con 0x7ff24c10fea0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.633+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19c4d0 con 0x7ff24c10fea0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.633+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a87f0 con 0x7ff24c10fea0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.633+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff244007e30 con 0x7ff24c10fea0 2024-05-08T06:04:46.635 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.633+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2440074e0 con 0x7ff24c10fea0 2024-05-08T06:04:46.636 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.634+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff230005330 con 0x7ff24c10fea0 2024-05-08T06:04:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.635+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7ff244003c00 con 0x7ff24c10fea0 2024-05-08T06:04:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.636+0000 7ff242ffd700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff2380b29f0 0x7ff2380b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.636+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9494+0+0 (secure 0 0 0) 0x7ff2440e38f0 con 0x7ff24c10fea0 2024-05-08T06:04:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.636+0000 7ff2518c7700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff2380b29f0 0x7ff2380b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:46.637 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.637+0000 7ff2518c7700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff2380b29f0 0x7ff2380b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff248005950 tx=0x7ff24800a300 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:46.639 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.639+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff24402a030 con 0x7ff24c10fea0 2024-05-08T06:04:46.855 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:46.855+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff230005620 con 0x7ff24c10fea0 2024-05-08T06:04:47.772 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.772+0000 7ff242ffd700 1 -- 172.21.15.3:0/4291003626 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v103) ==== 181+0+0 (secure 0 0 0) 0x7ff2440aaf90 con 0x7ff24c10fea0 2024-05-08T06:04:47.776 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff2380b29f0 msgr2=0x7ff2380b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff2380b29f0 0x7ff2380b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff248005950 tx=0x7ff24800a300 comp rx=0 tx=0).stop 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 msgr2=0x7ff24c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c19bae0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff24400b830 tx=0x7ff244004ce0 comp rx=0 tx=0).stop 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 shutdown_connections 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7ff24c10fea0 0x7ff24c19bae0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7ff2380b29f0 0x7ff2380b4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff24c0756d0 0x7ff24c1a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 --2- 172.21.15.3:0/4291003626 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff24c075ff0 0x7ff24c1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:47.777 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.776+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 >> 172.21.15.3:0/4291003626 conn(0x7ff24c07a600 msgr2=0x7ff24c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.777+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 shutdown_connections 2024-05-08T06:04:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:47.777+0000 7ff253b2b700 1 -- 172.21.15.3:0/4291003626 wait complete. 2024-05-08T06:04:47.778 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_metadata' removed 2024-05-08T06:04:48.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:47 smithi104 ceph-mon[34559]: pgmap v345: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:48.021 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:47 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/4291003626' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:48.106 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:47 smithi003 ceph-mon[30137]: pgmap v345: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:48.106 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:47 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/4291003626' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:48.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:47 smithi089 ceph-mon[34495]: pgmap v345: 129 pgs: 129 active+clean; 2.5 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-08T06:04:48.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:47 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/4291003626' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:48.434 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T06:04:48.695 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:49.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:48 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/4291003626' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:49.093 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:48 smithi003 ceph-mon[30137]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T06:04:49.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:48 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/4291003626' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:49.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:48 smithi089 ceph-mon[34495]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T06:04:49.270 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:48 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/4291003626' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:49.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:48 smithi104 ceph-mon[34559]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T06:04:49.872 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:49 smithi003 ceph-mon[30137]: pgmap v347: 97 pgs: 97 active+clean; 577 KiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:50.172 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:49 smithi104 ceph-mon[34559]: pgmap v347: 97 pgs: 97 active+clean; 577 KiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:50.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:49 smithi089 ceph-mon[34495]: pgmap v347: 97 pgs: 97 active+clean; 577 KiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 -- 172.21.15.3:0/2232703159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 msgr2=0x7f5ee41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/2232703159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee41103d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5edc00b0d0 tx=0x7f5edc00b3e0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 -- 172.21.15.3:0/2232703159 shutdown_connections 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/2232703159 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee41103d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/2232703159 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ee4103750 0x7f5ee4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/2232703159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 0x7f5ee4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 -- 172.21.15.3:0/2232703159 >> 172.21.15.3:0/2232703159 conn(0x7f5ee407a600 msgr2=0x7f5ee407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 -- 172.21.15.3:0/2232703159 shutdown_connections 2024-05-08T06:04:50.447 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.446+0000 7f5eea8aa700 1 -- 172.21.15.3:0/2232703159 wait complete. 2024-05-08T06:04:50.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 Processor -- start 2024-05-08T06:04:50.448 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 -- start start 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ee4103750 0x7f5ee40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 0x7f5ee4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4113620 con 0x7f5ee4108850 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41134a0 con 0x7f5ee4104100 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5eea8aa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41137a0 con 0x7f5ee4103750 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5ee8e47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5ee8e47700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee4076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:40840/0 (socket says 172.21.15.3:40840) 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.447+0000 7f5ee8e47700 1 -- 172.21.15.3:0/3195420600 learned_addr learned my addr 172.21.15.3:0/3195420600 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:50.450 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee3fff700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ee4103750 0x7f5ee40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee37fe700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 0x7f5ee4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee8e47700 1 -- 172.21.15.3:0/3195420600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ee4103750 msgr2=0x7f5ee40757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee8e47700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ee4103750 0x7f5ee40757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee8e47700 1 -- 172.21.15.3:0/3195420600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 msgr2=0x7f5ee4075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee8e47700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 0x7f5ee4075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee8e47700 1 -- 172.21.15.3:0/3195420600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee41a7f90 con 0x7f5ee4108850 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee37fe700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 0x7f5ee4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee8e47700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee4076220 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5edc007a00 tx=0x7f5edc007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01f070 con 0x7f5ee4108850 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5edc005520 con 0x7f5ee4108850 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee41a8220 con 0x7f5ee4108850 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc01b670 con 0x7f5ee4108850 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.448+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41a8670 con 0x7f5ee4108850 2024-05-08T06:04:50.451 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.450+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee406a190 con 0x7f5ee4108850 2024-05-08T06:04:50.452 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.451+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7f5edc022070 con 0x7f5ee4108850 2024-05-08T06:04:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.452+0000 7f5ee17fa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f5ecc0b29f0 0x7f5ecc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.452+0000 7f5ee3fff700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f5ecc0b29f0 0x7f5ecc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.452+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8969+0+0 (secure 0 0 0) 0x7f5edc00f020 con 0x7f5ee4108850 2024-05-08T06:04:50.453 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.452+0000 7f5ee3fff700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f5ecc0b29f0 0x7f5ecc0b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009bf0 tx=0x7f5ed4009260 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:50.457 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.457+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5edc029020 con 0x7f5ee4108850 2024-05-08T06:04:50.669 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.669+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ee402e510 con 0x7f5ee4108850 2024-05-08T06:04:50.786 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.786+0000 7f5ee17fa700 1 -- 172.21.15.3:0/3195420600 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v104) ==== 169+0+0 (secure 0 0 0) 0x7f5edc0ab170 con 0x7f5ee4108850 2024-05-08T06:04:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f5ecc0b29f0 msgr2=0x7f5ecc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f5ecc0b29f0 0x7f5ecc0b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4009bf0 tx=0x7f5ed4009260 comp rx=0 tx=0).stop 2024-05-08T06:04:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 msgr2=0x7f5ee4076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:50.789 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee4076220 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5edc007a00 tx=0x7f5edc007ae0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 shutdown_connections 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7f5ee4108850 0x7f5ee4076220 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7f5ecc0b29f0 0x7f5ecc0b4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ee4103750 0x7f5ee40757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 --2- 172.21.15.3:0/3195420600 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5ee4104100 0x7f5ee4075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 >> 172.21.15.3:0/3195420600 conn(0x7f5ee407a600 msgr2=0x7f5ee41016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 shutdown_connections 2024-05-08T06:04:50.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:50.789+0000 7f5eea8aa700 1 -- 172.21.15.3:0/3195420600 wait complete. 2024-05-08T06:04:50.791 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data' removed 2024-05-08T06:04:51.113 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:50 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3195420600' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:51.185 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:50 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3195420600' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:51.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:50 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3195420600' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:51.445 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T06:04:51.709 INFO:teuthology.orchestra.run.smithi003.stderr:Inferring config /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T06:04:51.788 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:51 smithi003 ceph-mon[30137]: pgmap v348: 97 pgs: 97 active+clean; 577 KiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:52.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:51 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/3195420600' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:52.091 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:51 smithi003 ceph-mon[30137]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T06:04:52.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:51 smithi089 ceph-mon[34495]: pgmap v348: 97 pgs: 97 active+clean; 577 KiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:52.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:51 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/3195420600' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:52.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:51 smithi089 ceph-mon[34495]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T06:04:52.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:51 smithi104 ceph-mon[34559]: pgmap v348: 97 pgs: 97 active+clean; 577 KiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:52.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:51 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/3195420600' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:52.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:51 smithi104 ceph-mon[34559]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 -- 172.21.15.3:0/2025193492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 msgr2=0x7fe71c107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 --2- 172.21.15.3:0/2025193492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c107020 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe7100099c0 tx=0x7fe710009cd0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 -- 172.21.15.3:0/2025193492 shutdown_connections 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 --2- 172.21.15.3:0/2025193492 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 0x7fe71c10f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 --2- 172.21.15.3:0/2025193492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 0x7fe71c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 --2- 172.21.15.3:0/2025193492 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c107020 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.479 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 -- 172.21.15.3:0/2025193492 >> 172.21.15.3:0/2025193492 conn(0x7fe71c079440 msgr2=0x7fe71c079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 -- 172.21.15.3:0/2025193492 shutdown_connections 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.476+0000 7fe7224ad700 1 -- 172.21.15.3:0/2025193492 wait complete. 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 Processor -- start 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 -- start start 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 0x7fe71c19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c19afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 0x7fe71c1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 -- --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c112590 con 0x7fe71c102fd0 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c112410 con 0x7fe71c107700 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe7224ad700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c112710 con 0x7fe71c102620 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c19afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe71b7fe700 1 --2- >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c19afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.3:3300/0 says I am v2:172.21.15.3:47574/0 (socket says 172.21.15.3:47574) 2024-05-08T06:04:53.480 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.477+0000 7fe71b7fe700 1 -- 172.21.15.3:0/1590656595 learned_addr learned my addr 172.21.15.3:0/1590656595 (peer_addr_for_me v2:172.21.15.3:0/0) 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71bfff700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 0x7fe71c19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe720a4a700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 0x7fe71c1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71b7fe700 1 -- 172.21.15.3:0/1590656595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 msgr2=0x7fe71c19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71b7fe700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 0x7fe71c19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71b7fe700 1 -- 172.21.15.3:0/1590656595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 msgr2=0x7fe71c1a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71b7fe700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 0x7fe71c1a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71b7fe700 1 -- 172.21.15.3:0/1590656595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c1a6050 con 0x7fe71c102fd0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71bfff700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 0x7fe71c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe720a4a700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 0x7fe71c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe71b7fe700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c19afd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe710006010 tx=0x7fe710005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710027070 con 0x7fe71c102fd0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe710004020 con 0x7fe71c102fd0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c1a6220 con 0x7fe71c102fd0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710022d00 con 0x7fe71c102fd0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.478+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a6670 con 0x7fe71c102fd0 2024-05-08T06:04:53.481 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.479+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c06a190 con 0x7fe71c102fd0 2024-05-08T06:04:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.481+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 4 ==== mgrmap(e 26) ==== 149782+0+0 (secure 0 0 0) 0x7fe71002a070 con 0x7fe71c102fd0 2024-05-08T06:04:53.482 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.482+0000 7fe7197fa700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe7040b29f0 0x7fe7040b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T06:04:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.482+0000 7fe71bfff700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe7040b29f0 0x7fe7040b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T06:04:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.482+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8410+0+0 (secure 0 0 0) 0x7fe7100e3410 con 0x7fe71c102fd0 2024-05-08T06:04:53.483 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.482+0000 7fe71bfff700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe7040b29f0 0x7fe7040b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00bcf0 tx=0x7fe70c00b2c0 comp rx=0 tx=0).ready entity=mgr.15129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T06:04:53.485 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.485+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe710013070 con 0x7fe71c102fd0 2024-05-08T06:04:53.699 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.698+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 --> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe71c02e510 con 0x7fe71c102fd0 2024-05-08T06:04:53.790 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.790+0000 7fe7197fa700 1 -- 172.21.15.3:0/1590656595 <== mon.0 v2:172.21.15.3:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v105) ==== 178+0+0 (secure 0 0 0) 0x7fe7100aae90 con 0x7fe71c102fd0 2024-05-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe7040b29f0 msgr2=0x7fe7040b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe7040b29f0 0x7fe7040b4eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00bcf0 tx=0x7fe70c00b2c0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 msgr2=0x7fe71c19afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c19afd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe710006010 tx=0x7fe710005990 comp rx=0 tx=0).stop 2024-05-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 shutdown_connections 2024-05-08T06:04:53.793 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe71c107700 0x7fe71c1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:6800/3346608619,v1:172.21.15.3:6801/3346608619] conn(0x7fe7040b29f0 0x7fe7040b4eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe71c102620 0x7fe71c19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 --2- 172.21.15.3:0/1590656595 >> [v2:172.21.15.3:3300/0,v1:172.21.15.3:6789/0] conn(0x7fe71c102fd0 0x7fe71c19afd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 >> 172.21.15.3:0/1590656595 conn(0x7fe71c079440 msgr2=0x7fe71c100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 shutdown_connections 2024-05-08T06:04:53.794 INFO:teuthology.orchestra.run.smithi003.stderr:2024-05-08T06:04:53.793+0000 7fe7224ad700 1 -- 172.21.15.3:0/1590656595 wait complete. 2024-05-08T06:04:53.795 INFO:teuthology.orchestra.run.smithi003.stderr:pool 'cephfs_data_ec' removed 2024-05-08T06:04:54.065 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:53 smithi089 ceph-mon[34495]: pgmap v350: 33 pgs: 33 active+clean; 577 KiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:54.065 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:53 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1590656595' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:54.120 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:53 smithi003 ceph-mon[30137]: pgmap v350: 33 pgs: 33 active+clean; 577 KiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:54.120 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:53 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1590656595' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:53 smithi104 ceph-mon[34559]: pgmap v350: 33 pgs: 33 active+clean; 577 KiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T06:04:54.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:53 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1590656595' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T06:04:54.419 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:04:54.449 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:04:54.483 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:04:54.515 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T06:04:54.515 DEBUG:teuthology.orchestra.run.smithi003:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:04:54.533 DEBUG:teuthology.orchestra.run.smithi089:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:04:54.550 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T06:04:54.573 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T06:04:54.573 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T06:04:54.574 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a 2024-05-08T06:04:54.659 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:54 smithi003 systemd[1]: Stopping Ceph mon.a for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:04:55.159 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:54 smithi003 ceph-mon[30137]: from='client.? 172.21.15.3:0/1590656595' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:55.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:54 smithi003 ceph-mon[30137]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T06:04:55.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a[30133]: 2024-05-08T06:04:54.899+0000 7fec911df700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:04:55.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a[30133]: 2024-05-08T06:04:54.899+0000 7fec911df700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T06:04:55.160 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:54 smithi003 podman[99172]: 2024-05-08 06:04:54.931086262 +0000 UTC m=+0.154120764 container died 25a2aff035ee0634c0905d1a3098821605dac2b066ea8bcd6b0f9e26cc20dc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-05-08T06:04:55.262 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:54 smithi089 ceph-mon[34495]: from='client.? 172.21.15.3:0/1590656595' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:55.263 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:54 smithi089 ceph-mon[34495]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T06:04:55.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:54 smithi104 ceph-mon[34559]: from='client.? 172.21.15.3:0/1590656595' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T06:04:55.271 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:54 smithi104 ceph-mon[34559]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T06:04:55.559 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:55 smithi003 podman[99172]: 2024-05-08 06:04:55.317033745 +0000 UTC m=+0.540068255 container cleanup 25a2aff035ee0634c0905d1a3098821605dac2b066ea8bcd6b0f9e26cc20dc3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-05-08T06:04:55.559 INFO:journalctl@ceph.mon.a.smithi003.stdout:May 08 06:04:55 smithi003 bash[99172]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-a 2024-05-08T06:04:55.695 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T06:04:55.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:04:55.744 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T06:04:55.744 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T06:04:55.744 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.b 2024-05-08T06:04:56.037 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:55 smithi089 systemd[1]: Stopping Ceph mon.b for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:04:56.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:56 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-b[34491]: 2024-05-08T06:04:56.037+0000 7f602d189700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:04:56.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:56 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-b[34491]: 2024-05-08T06:04:56.037+0000 7f602d189700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T06:04:56.380 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:56 smithi089 podman[71242]: 2024-05-08 06:04:56.088697861 +0000 UTC m=+0.148152853 container died 2495a7dc70cb4e7615b4125b784207509fb7f0bcd3894bdbf7e7aa1b9fcba34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T06:04:56.739 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:56 smithi089 podman[71242]: 2024-05-08 06:04:56.479634458 +0000 UTC m=+0.539089450 container cleanup 2495a7dc70cb4e7615b4125b784207509fb7f0bcd3894bdbf7e7aa1b9fcba34a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:04:56.843 INFO:journalctl@ceph.mon.b.smithi089.stdout:May 08 06:04:56 smithi089 bash[71242]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-b 2024-05-08T06:04:56.895 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T06:04:56.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:04:56.950 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T06:04:56.950 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T06:04:56.950 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.c 2024-05-08T06:04:57.262 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:56 smithi104 systemd[1]: Stopping Ceph mon.c for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:04:57.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c[34555]: 2024-05-08T06:04:57.262+0000 7f3dd39f7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:04:57.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c[34555]: 2024-05-08T06:04:57.262+0000 7f3dd39f7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T06:04:57.521 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:57 smithi104 podman[71160]: 2024-05-08 06:04:57.467132157 +0000 UTC m=+0.322689744 container died 8f7693d5b131b0f4f4f7c7310dffec012fe1eb9f21366f4cf9b393b4d8a4ab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T06:04:58.130 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:57 smithi104 podman[71160]: 2024-05-08 06:04:57.871242398 +0000 UTC m=+0.726799983 container cleanup 8f7693d5b131b0f4f4f7c7310dffec012fe1eb9f21366f4cf9b393b4d8a4ab39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T06:04:58.130 INFO:journalctl@ceph.mon.c.smithi104.stdout:May 08 06:04:57 smithi104 bash[71160]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mon-c 2024-05-08T06:04:58.288 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T06:04:58.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:04:58.339 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T06:04:58.339 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T06:04:58.339 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x 2024-05-08T06:04:58.626 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 06:04:58 smithi003 systemd[1]: Stopping Ceph mgr.x for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:04:58.898 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 06:04:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x[30370]: 2024-05-08T06:04:58.663+0000 7fb92b4af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:04:58.898 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 06:04:58 smithi003 podman[99295]: 2024-05-08 06:04:58.695196444 +0000 UTC m=+0.156281846 container died a09ad788764b7ffe0dc90992ea608c3371dacc7274f212131c9bb68f5996a44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:04:59.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 06:04:59 smithi003 podman[99295]: 2024-05-08 06:04:59.054867838 +0000 UTC m=+0.515953241 container cleanup a09ad788764b7ffe0dc90992ea608c3371dacc7274f212131c9bb68f5996a44f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T06:04:59.160 INFO:journalctl@ceph.mgr.x.smithi003.stdout:May 08 06:04:59 smithi003 bash[99295]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-x 2024-05-08T06:04:59.483 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T06:04:59.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:04:59.523 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T06:04:59.523 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T06:04:59.524 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.y 2024-05-08T06:04:59.837 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 06:04:59 smithi089 systemd[1]: Stopping Ceph mgr.y for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:00.135 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 06:04:59 smithi089 podman[71363]: 2024-05-08 06:04:59.838135104 +0000 UTC m=+0.125453487 container died bf127d7b51728de569e5cda7cc830d07a275c1bb8edd20fceb019d7b6db8070f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:00.486 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 06:05:00 smithi089 podman[71363]: 2024-05-08 06:05:00.226706615 +0000 UTC m=+0.514024990 container cleanup bf127d7b51728de569e5cda7cc830d07a275c1bb8edd20fceb019d7b6db8070f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-08T06:05:00.486 INFO:journalctl@ceph.mgr.y.smithi089.stdout:May 08 06:05:00 smithi089 bash[71363]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-y 2024-05-08T06:05:00.634 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T06:05:00.666 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:00.667 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T06:05:00.667 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T06:05:00.667 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.z 2024-05-08T06:05:00.996 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 06:05:00 smithi104 systemd[1]: Stopping Ceph mgr.z for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:01.271 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 06:05:00 smithi104 podman[71281]: 2024-05-08 06:05:00.996583631 +0000 UTC m=+0.137981849 container died 373bbb82490d2c3dbbdfcdf609f9d085ebabef6db75f82c3cf76f8b6880dde00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-08T06:05:01.627 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 06:05:01 smithi104 podman[71281]: 2024-05-08 06:05:01.426819842 +0000 UTC m=+0.568218057 container cleanup 373bbb82490d2c3dbbdfcdf609f9d085ebabef6db75f82c3cf76f8b6880dde00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:01.627 INFO:journalctl@ceph.mgr.z.smithi104.stdout:May 08 06:05:01 smithi104 bash[71281]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mgr-z 2024-05-08T06:05:01.844 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T06:05:01.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:01.878 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T06:05:01.879 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T06:05:01.879 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.0 2024-05-08T06:05:02.160 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:01 smithi003 systemd[1]: Stopping Ceph osd.0 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:02.660 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0[44525]: 2024-05-08T06:05:02.218+0000 7fa42ab1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:02.660 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0[44525]: 2024-05-08T06:05:02.218+0000 7fa42ab1f700 -1 osd.0 105 *** Got signal Terminated *** 2024-05-08T06:05:02.660 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0[44525]: 2024-05-08T06:05:02.218+0000 7fa42ab1f700 -1 osd.0 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:08.019 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:07 smithi003 podman[99430]: 2024-05-08 06:05:07.69035325 +0000 UTC m=+5.613789376 container died 0310133f16c3d6a14ed7fcb476f02fe945ab794c14409e0c342c8a999dac8aa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T06:05:08.343 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 podman[99430]: 2024-05-08 06:05:08.101311508 +0000 UTC m=+6.024747634 container cleanup 0310133f16c3d6a14ed7fcb476f02fe945ab794c14409e0c342c8a999dac8aa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-08T06:05:08.344 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 bash[99430]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0 2024-05-08T06:05:08.660 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 podman[99447]: 2024-05-08 06:05:08.344277302 +0000 UTC m=+0.646628417 container remove 0310133f16c3d6a14ed7fcb476f02fe945ab794c14409e0c342c8a999dac8aa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-05-08T06:05:08.964 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 podman[99540]: 2024-05-08 06:05:08.575370379 +0000 UTC m=+0.018228266 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:08.964 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 podman[99540]: 2024-05-08 06:05:08.695263342 +0000 UTC m=+0.138121212 container create c58dcfccdff5ec2d8db59a2fd78f3158c9ccf13694d2310fa41dc53a930abab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:08.965 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 podman[99540]: 2024-05-08 06:05:08.964537312 +0000 UTC m=+0.407395196 container init c58dcfccdff5ec2d8db59a2fd78f3158c9ccf13694d2310fa41dc53a930abab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:09.274 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:08 smithi003 podman[99540]: 2024-05-08 06:05:08.968884571 +0000 UTC m=+0.411742442 container start c58dcfccdff5ec2d8db59a2fd78f3158c9ccf13694d2310fa41dc53a930abab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-08T06:05:09.274 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:09 smithi003 podman[99540]: 2024-05-08 06:05:09.08160644 +0000 UTC m=+0.524464313 container attach c58dcfccdff5ec2d8db59a2fd78f3158c9ccf13694d2310fa41dc53a930abab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T06:05:09.643 INFO:journalctl@ceph.osd.0.smithi003.stdout:May 08 06:05:09 smithi003 podman[99540]: 2024-05-08 06:05:09.275357335 +0000 UTC m=+0.718215208 container died c58dcfccdff5ec2d8db59a2fd78f3158c9ccf13694d2310fa41dc53a930abab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-05-08T06:05:09.946 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T06:05:09.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:09.991 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T06:05:09.991 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T06:05:09.991 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.1 2024-05-08T06:05:10.300 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:10 smithi089 systemd[1]: Stopping Ceph osd.1 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:10.762 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-08T06:05:10.299+0000 7f34956ce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:10.762 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-08T06:05:10.299+0000 7f34956ce700 -1 osd.1 105 *** Got signal Terminated *** 2024-05-08T06:05:10.763 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1[39140]: 2024-05-08T06:05:10.299+0000 7f34956ce700 -1 osd.1 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:16.012 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:15 smithi089 podman[71488]: 2024-05-08 06:05:15.761215402 +0000 UTC m=+5.577825839 container died 2294fb74133e285f8f5800ae59b65cf20d6228bcd1d87c2e843e54f9e5f265ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-08T06:05:16.385 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 podman[71488]: 2024-05-08 06:05:16.134340764 +0000 UTC m=+5.950951206 container cleanup 2294fb74133e285f8f5800ae59b65cf20d6228bcd1d87c2e843e54f9e5f265ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:16.385 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 bash[71488]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1 2024-05-08T06:05:16.702 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 podman[71505]: 2024-05-08 06:05:16.386196816 +0000 UTC m=+0.621036654 container remove 2294fb74133e285f8f5800ae59b65cf20d6228bcd1d87c2e843e54f9e5f265ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:16.964 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 podman[71598]: 2024-05-08 06:05:16.606218143 +0000 UTC m=+0.017810422 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:16.964 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 podman[71598]: 2024-05-08 06:05:16.703949916 +0000 UTC m=+0.115542225 container create c9ba9ffb0c98b2abb4839d8ed19d45784558ca58957a1c9001c8e2bca580f440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True) 2024-05-08T06:05:17.263 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 podman[71598]: 2024-05-08 06:05:16.965413659 +0000 UTC m=+0.377005926 container init c9ba9ffb0c98b2abb4839d8ed19d45784558ca58957a1c9001c8e2bca580f440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:17.263 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:16 smithi089 podman[71598]: 2024-05-08 06:05:16.968852935 +0000 UTC m=+0.380445203 container start c9ba9ffb0c98b2abb4839d8ed19d45784558ca58957a1c9001c8e2bca580f440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-08T06:05:17.263 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:17 smithi089 podman[71598]: 2024-05-08 06:05:17.057319492 +0000 UTC m=+0.468911760 container attach c9ba9ffb0c98b2abb4839d8ed19d45784558ca58957a1c9001c8e2bca580f440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:05:17.534 INFO:journalctl@ceph.osd.1.smithi089.stdout:May 08 06:05:17 smithi089 podman[71598]: 2024-05-08 06:05:17.264207187 +0000 UTC m=+0.675799447 container died c9ba9ffb0c98b2abb4839d8ed19d45784558ca58957a1c9001c8e2bca580f440 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:17.916 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T06:05:17.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:17.954 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T06:05:17.954 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T06:05:17.954 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.2 2024-05-08T06:05:18.271 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:17 smithi104 systemd[1]: Stopping Ceph osd.2 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:18.771 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2[39728]: 2024-05-08T06:05:18.329+0000 7ff975607700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:18.771 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2[39728]: 2024-05-08T06:05:18.329+0000 7ff975607700 -1 osd.2 105 *** Got signal Terminated *** 2024-05-08T06:05:18.771 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2[39728]: 2024-05-08T06:05:18.329+0000 7ff975607700 -1 osd.2 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:24.177 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:23 smithi104 podman[71404]: 2024-05-08 06:05:23.876130596 +0000 UTC m=+5.732557747 container died 54e755a3a40de379fe9f72c7252a1231c3058a74f94f0dbcdbf5d2367b464af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-08T06:05:24.521 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:24 smithi104 podman[71404]: 2024-05-08 06:05:24.321101894 +0000 UTC m=+6.177529053 container cleanup 54e755a3a40de379fe9f72c7252a1231c3058a74f94f0dbcdbf5d2367b464af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T06:05:24.521 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:24 smithi104 bash[71404]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2 2024-05-08T06:05:24.994 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:24 smithi104 podman[71422]: 2024-05-08 06:05:24.555583179 +0000 UTC m=+0.676259286 container remove 54e755a3a40de379fe9f72c7252a1231c3058a74f94f0dbcdbf5d2367b464af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-08T06:05:25.245 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:24 smithi104 podman[71515]: 2024-05-08 06:05:24.898325143 +0000 UTC m=+0.017621767 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:25.245 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:25 smithi104 podman[71515]: 2024-05-08 06:05:25.006481394 +0000 UTC m=+0.125778001 container create bec833c970f13a890296747aa7ae3c87d32c9f7677d2d630adb4a8ee8f6e92a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:25.245 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:25 smithi104 podman[71515]: 2024-05-08 06:05:25.242723448 +0000 UTC m=+0.362020049 container init bec833c970f13a890296747aa7ae3c87d32c9f7677d2d630adb4a8ee8f6e92a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:25.245 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:25 smithi104 podman[71515]: 2024-05-08 06:05:25.245734253 +0000 UTC m=+0.365030865 container start bec833c970f13a890296747aa7ae3c87d32c9f7677d2d630adb4a8ee8f6e92a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-08T06:05:25.521 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:25 smithi104 podman[71515]: 2024-05-08 06:05:25.3430252 +0000 UTC m=+0.462321806 container attach bec833c970f13a890296747aa7ae3c87d32c9f7677d2d630adb4a8ee8f6e92a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T06:05:25.896 INFO:journalctl@ceph.osd.2.smithi104.stdout:May 08 06:05:25 smithi104 podman[71515]: 2024-05-08 06:05:25.540758666 +0000 UTC m=+0.660055279 container died bec833c970f13a890296747aa7ae3c87d32c9f7677d2d630adb4a8ee8f6e92a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-08T06:05:26.224 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T06:05:26.262 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:26.262 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T06:05:26.262 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T06:05:26.262 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.3 2024-05-08T06:05:26.588 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:26 smithi003 systemd[1]: Stopping Ceph osd.3 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:26.910 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:26.587+0000 7f82f9c96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:26.910 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:26.587+0000 7f82f9c96700 -1 osd.3 105 *** Got signal Terminated *** 2024-05-08T06:05:26.910 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:26 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:26.587+0000 7f82f9c96700 -1 osd.3 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:28.712 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:28 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:28.444+0000 7f82f2282700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:03.302313+0000 front 2024-05-08T06:05:03.302445+0000 (oldest deadline 2024-05-08T06:05:28.002520+0000) 2024-05-08T06:05:29.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:29 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:29.131+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:29.909 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:29 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:29.456+0000 7f82f2282700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:03.302313+0000 front 2024-05-08T06:05:03.302445+0000 (oldest deadline 2024-05-08T06:05:28.002520+0000) 2024-05-08T06:05:30.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:30 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:30.156+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:30.909 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:30 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:30.496+0000 7f82f2282700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:03.302313+0000 front 2024-05-08T06:05:03.302445+0000 (oldest deadline 2024-05-08T06:05:28.002520+0000) 2024-05-08T06:05:31.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:30 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:30.753+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:31.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:30.988+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:31.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:30.921+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:31.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:31.111+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:31.910 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:31 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3[51457]: 2024-05-08T06:05:31.501+0000 7f82f2282700 -1 osd.3 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:03.302313+0000 front 2024-05-08T06:05:03.302445+0000 (oldest deadline 2024-05-08T06:05:28.002520+0000) 2024-05-08T06:05:32.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:31 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:31.745+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:32.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:31.980+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:32.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:31.955+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:32.329 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:32 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:32.133+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:32.329 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:31 smithi003 podman[99647]: 2024-05-08 06:05:31.966031191 +0000 UTC m=+5.502306832 container died 1fe2b9afff9db765346c8a1c142caa2e090fa59e56db8266a09db134ae60b06a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T06:05:32.659 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:32 smithi003 podman[99647]: 2024-05-08 06:05:32.420877119 +0000 UTC m=+5.957152757 container cleanup 1fe2b9afff9db765346c8a1c142caa2e090fa59e56db8266a09db134ae60b06a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-05-08T06:05:32.660 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:32 smithi003 bash[99647]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3 2024-05-08T06:05:33.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:32 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:32.735+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:33.061 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:32 smithi003 podman[99676]: 2024-05-08 06:05:32.722201151 +0000 UTC m=+0.753957008 container remove 1fe2b9afff9db765346c8a1c142caa2e090fa59e56db8266a09db134ae60b06a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-08T06:05:33.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:33.019+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:33.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:32.972+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:33.358 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:33 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:33.094+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:33.358 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:33 smithi003 podman[99771]: 2024-05-08 06:05:32.96548142 +0000 UTC m=+0.018681157 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:33.358 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:33 smithi003 podman[99771]: 2024-05-08 06:05:33.098179893 +0000 UTC m=+0.151379657 container create 1adf3ee37df1e474b0b5e06399e4632c65efe4b503c9487dcd4fb29ff86d9e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:33.660 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:33 smithi003 podman[99771]: 2024-05-08 06:05:33.358971773 +0000 UTC m=+0.412171498 container init 1adf3ee37df1e474b0b5e06399e4632c65efe4b503c9487dcd4fb29ff86d9e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-08T06:05:33.660 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:33 smithi003 podman[99771]: 2024-05-08 06:05:33.36312718 +0000 UTC m=+0.416326904 container start 1adf3ee37df1e474b0b5e06399e4632c65efe4b503c9487dcd4fb29ff86d9e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-08T06:05:33.660 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:33 smithi003 podman[99771]: 2024-05-08 06:05:33.467341148 +0000 UTC m=+0.520540901 container attach 1adf3ee37df1e474b0b5e06399e4632c65efe4b503c9487dcd4fb29ff86d9e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507) 2024-05-08T06:05:33.977 INFO:journalctl@ceph.osd.3.smithi003.stdout:May 08 06:05:33 smithi003 podman[99771]: 2024-05-08 06:05:33.672115753 +0000 UTC m=+0.725315469 container died 1adf3ee37df1e474b0b5e06399e4632c65efe4b503c9487dcd4fb29ff86d9e9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:34.254 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:34 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:34.102+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:34.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:33 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:33.763+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:34.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:34.003+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:34.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:33.924+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:34.284 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T06:05:34.327 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:34.328 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T06:05:34.328 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T06:05:34.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.4 2024-05-08T06:05:34.656 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:34 smithi089 systemd[1]: Stopping Ceph osd.4 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:35.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:34 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:34.792+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:35.013 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:34 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T06:05:34.656+0000 7f87a0257700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:35.013 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:34 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T06:05:34.656+0000 7f87a0257700 -1 osd.4 105 *** Got signal Terminated *** 2024-05-08T06:05:35.013 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:34 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T06:05:34.656+0000 7f87a0257700 -1 osd.4 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:35.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:35 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:35.045+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:34.957+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:35.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:35 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:35.076+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:36.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:35 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:35.832+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:36.270 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:36 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:36.060+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:36.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:36 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:36.000+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:36.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:36 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:36.036+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:36.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:36 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:36.046+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:37.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:36 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:36.794+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:37.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:37 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:37.029+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:37.446 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:37 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:37.034+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:37.446 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:37 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:37.070+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:37.447 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:37 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:37.025+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:38.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:37 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:37.748+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:38.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:37 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:37.748+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:38.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:38.008+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:38.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:38.077+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:38.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:38.047+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:38.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:38 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:38.065+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:38.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:38 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:38.065+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:38.512 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:38 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T06:05:38.102+0000 7f879c84b700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.201606+0000 front 2024-05-08T06:05:13.201630+0000 (oldest deadline 2024-05-08T06:05:37.301734+0000) 2024-05-08T06:05:39.082 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:38 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:38.772+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:39.083 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:38 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:38.772+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:39.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:39.064+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:39.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:39.048+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:39.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:38.962+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:39.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:38 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:38.962+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:39.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:39.040+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:39.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:39 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:39.040+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:39.512 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4[46086]: 2024-05-08T06:05:39.083+0000 7f879c84b700 -1 osd.4 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.201606+0000 front 2024-05-08T06:05:13.201630+0000 (oldest deadline 2024-05-08T06:05:37.301734+0000) 2024-05-08T06:05:39.997 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:39.687+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:39.997 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:39.817+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:39.997 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:39 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:39.817+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:40.262 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:39 smithi089 podman[71716]: 2024-05-08 06:05:39.997884577 +0000 UTC m=+5.465602809 container died 9f757a5eba896aad20ed2f44209ef20ea4ed13b457181896c8a3f8d52a03d415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-08T06:05:40.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:40 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:40.092+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:40.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:39.955+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:40.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:39 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:39.955+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:40.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:40 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:40.011+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:40.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:40 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:40.049+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:40.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:40 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:40.049+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:40.616 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:40 smithi089 podman[71716]: 2024-05-08 06:05:40.391172696 +0000 UTC m=+5.858890927 container cleanup 9f757a5eba896aad20ed2f44209ef20ea4ed13b457181896c8a3f8d52a03d415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-08T06:05:40.617 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:40 smithi089 bash[71716]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4 2024-05-08T06:05:40.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:40 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:40.549+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:41.012 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:40 smithi089 podman[71733]: 2024-05-08 06:05:40.617609017 +0000 UTC m=+0.612523212 container remove 9f757a5eba896aad20ed2f44209ef20ea4ed13b457181896c8a3f8d52a03d415 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:41.013 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:40 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:40.720+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:41.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:40 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:40.833+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:41.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:40 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:40.834+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:41.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:41.075+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:41.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:40 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:40.922+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:41.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:40 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:40.922+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:41.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:41.053+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:41.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:41.053+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:41.346 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:41 smithi089 podman[71823]: 2024-05-08 06:05:40.967987335 +0000 UTC m=+0.017987608 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:41.346 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:41 smithi089 podman[71823]: 2024-05-08 06:05:41.077491065 +0000 UTC m=+0.127491361 container create 0194fdc4aa8bfeedf43c7f8a68a39fa1940766ccbf5870a5e8c23aa4d80f3c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:41.346 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:41 smithi089 podman[71823]: 2024-05-08 06:05:41.347415776 +0000 UTC m=+0.397416022 container init 0194fdc4aa8bfeedf43c7f8a68a39fa1940766ccbf5870a5e8c23aa4d80f3c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f) 2024-05-08T06:05:41.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:41.085+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:41.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:41.085+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:41.644 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:41 smithi089 podman[71823]: 2024-05-08 06:05:41.350564638 +0000 UTC m=+0.400564881 container start 0194fdc4aa8bfeedf43c7f8a68a39fa1940766ccbf5870a5e8c23aa4d80f3c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-08T06:05:41.644 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:41 smithi089 podman[71823]: 2024-05-08 06:05:41.448291169 +0000 UTC m=+0.498291419 container attach 0194fdc4aa8bfeedf43c7f8a68a39fa1940766ccbf5870a5e8c23aa4d80f3c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T06:05:41.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:41 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:41.547+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:41.935 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:41.712+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:41.935 INFO:journalctl@ceph.osd.4.smithi089.stdout:May 08 06:05:41 smithi089 podman[71823]: 2024-05-08 06:05:41.64603669 +0000 UTC m=+0.696036940 container died 0194fdc4aa8bfeedf43c7f8a68a39fa1940766ccbf5870a5e8c23aa4d80f3c96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-08T06:05:41.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:41.812+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:41.935 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:41 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:41.812+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:42.265 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T06:05:42.270 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:42.100+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:42.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:41.927+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:42.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:41 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:41.927+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:42.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:42.053+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:42.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:42.053+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:42.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:42.303 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T06:05:42.303 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T06:05:42.303 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.5 2024-05-08T06:05:42.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:42.075+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:42.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:42.075+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:42.622 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:42 smithi104 systemd[1]: Stopping Ceph osd.5 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:42.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:42.522+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:42.932 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:42.622+0000 7f244366a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:42.932 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:42.622+0000 7f244366a700 -1 osd.5 105 *** Got signal Terminated *** 2024-05-08T06:05:42.932 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:42.622+0000 7f244366a700 -1 osd.5 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:43.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:42 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:42.737+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:43.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:42 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:42.835+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:43.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:42 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:42.835+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:43.270 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:43.058+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:43.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:42.932+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:43.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:42 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:42.932+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:43.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:43.086+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:43.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:43.086+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:43.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:43.094+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:43.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:43.094+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:43.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:43 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:43.525+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:44.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:43.746+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:44.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:43.874+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:44.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:43 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:43.874+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:44.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:43.955+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:44.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:43 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:43.955+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:44.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:44.096+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:44.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:44.097+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:44.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:44.097+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:44.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:44.108+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:44.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:44.108+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:44.910 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:44.569+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:45.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:44.753+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:45.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:44.919+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:45.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:44 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:44.920+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:45.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:44.935+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:45.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:44 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:44.935+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:45.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:45.141+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:45.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:45.141+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:45.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:45.053+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:45.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:45.053+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:45.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:45.099+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:45.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:45.099+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:45.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:45 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:45.605+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:46.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:45 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:45.736+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:46.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:45 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:45.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:46.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:45 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:45.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:46.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:45 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:45.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:46.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:45.945+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:46.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:45.945+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:46.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:45 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:45.945+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:18.248522+0000 front 2024-05-08T06:05:18.248539+0000 (oldest deadline 2024-05-08T06:05:45.249075+0000) 2024-05-08T06:05:46.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:46.127+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:46.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:46.127+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:46.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:46.010+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:46.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:46.010+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:46.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:46.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:46.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:46.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:46.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:46.653+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:46.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:46 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:46.653+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:47.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:46 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:46.741+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:47.013 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:46 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:46.741+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:47.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:46 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:46.963+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:47.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:46 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:46.963+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:47.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:46 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:46.963+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:47.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:46.916+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:07.147271+0000 front 2024-05-08T06:05:07.147140+0000 (oldest deadline 2024-05-08T06:05:30.047097+0000) 2024-05-08T06:05:47.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:46.916+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.748165+0000 front 2024-05-08T06:05:14.748338+0000 (oldest deadline 2024-05-08T06:05:38.248086+0000) 2024-05-08T06:05:47.271 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:46 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5[46492]: 2024-05-08T06:05:46.916+0000 7f243bc56700 -1 osd.5 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:18.248522+0000 front 2024-05-08T06:05:18.248539+0000 (oldest deadline 2024-05-08T06:05:45.249075+0000) 2024-05-08T06:05:47.272 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:47.102+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:47.272 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:47.102+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:47.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:47.019+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:47.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:47.019+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:47.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:47 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:47.019+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:47.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:47.073+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:47.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:47.073+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:48.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:47 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:47.733+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:48.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:47 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:47.733+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:48.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:47 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:47.939+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:48.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:47 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:47.939+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:48.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:47 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:47.939+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:48.109 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:47.694+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:48.109 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:47 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:47.694+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:48.248 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:47 smithi104 podman[71634]: 2024-05-08 06:05:47.947335856 +0000 UTC m=+5.442916621 container died d93814719372989b8654db4a32af866925f9f1a1d3be99a5d2797b3d3c3f8c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:05:48.249 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:48.103+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:48.249 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:48.103+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:48.249 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:48.004+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:48.249 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:48.004+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:48.249 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:48.004+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:48.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:48 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:48.109+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:48.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:48 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:48.109+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:48.521 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:48 smithi104 podman[71634]: 2024-05-08 06:05:48.347483974 +0000 UTC m=+5.843064739 container cleanup d93814719372989b8654db4a32af866925f9f1a1d3be99a5d2797b3d3c3f8c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-08T06:05:48.521 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:48 smithi104 bash[71634]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5 2024-05-08T06:05:48.965 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:48 smithi104 podman[71651]: 2024-05-08 06:05:48.606987358 +0000 UTC m=+0.658967822 container remove d93814719372989b8654db4a32af866925f9f1a1d3be99a5d2797b3d3c3f8c6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-08T06:05:49.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:48.739+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:49.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:48.739+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:49.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:48.936+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:49.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:48.936+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:49.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:48 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:48.936+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:49.070 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:48 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:48.673+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:49.070 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:48 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:48.673+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:49.122+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:49.122+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:48 smithi104 podman[71737]: 2024-05-08 06:05:48.966010639 +0000 UTC m=+0.111056301 container create 5880b1f42653112c59b246efa4b76f4ff2c6954829e42e00d6d378b3e2833fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:48 smithi104 podman[71737]: 2024-05-08 06:05:48.873031383 +0000 UTC m=+0.018077046 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:48.964+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:48.964+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:49.219 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:48 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:48.965+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:49.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:49.070+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:49.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:49.070+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:49.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:49.070+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:49.514 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:49 smithi104 podman[71737]: 2024-05-08 06:05:49.219500865 +0000 UTC m=+0.364546528 container init 5880b1f42653112c59b246efa4b76f4ff2c6954829e42e00d6d378b3e2833fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:49.514 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:49 smithi104 podman[71737]: 2024-05-08 06:05:49.222604032 +0000 UTC m=+0.367649699 container start 5880b1f42653112c59b246efa4b76f4ff2c6954829e42e00d6d378b3e2833fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:49.515 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:49 smithi104 podman[71737]: 2024-05-08 06:05:49.310274894 +0000 UTC m=+0.455320583 container attach 5880b1f42653112c59b246efa4b76f4ff2c6954829e42e00d6d378b3e2833fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T06:05:49.771 INFO:journalctl@ceph.osd.5.smithi104.stdout:May 08 06:05:49 smithi104 podman[71737]: 2024-05-08 06:05:49.515934002 +0000 UTC m=+0.660979665 container died 5880b1f42653112c59b246efa4b76f4ff2c6954829e42e00d6d378b3e2833fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:05:49.976 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:49 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:49.709+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:49.976 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:49 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:49.709+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:50.061 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:49.685+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:50.061 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:49.685+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:50.110 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:49.931+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:50.110 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:49.931+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:50.110 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:49 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:49.931+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:50.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T06:05:50.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:50.236 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T06:05:50.236 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T06:05:50.236 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.6 2024-05-08T06:05:50.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:49 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:49.977+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:50.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:49 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:49.977+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:50.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:49 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:49.977+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:50.410 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:50 smithi003 systemd[1]: Stopping Ceph osd.6 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:50.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:50.061+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:50.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:50.061+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:50.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:50.061+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:50.520 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:50.110+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:50.520 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:50.110+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:50.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:50.564+0000 7fcc2be44700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:50.910 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:50.564+0000 7fcc2be44700 -1 osd.6 105 *** Got signal Terminated *** 2024-05-08T06:05:50.910 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:50.564+0000 7fcc2be44700 -1 osd.6 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:50.910 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:50.646+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:50.910 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:50 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:50.646+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:51.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:50.757+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:51.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:50 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:50.757+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:51.012 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:51.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:51.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:51.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:51.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:51.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:51.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:51.066+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:51.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:51.066+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:51.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:50.941+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:51.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:50.941+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:51.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:50 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:50.941+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:51.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:51 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:51.094+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:51.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:51 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:51.094+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:51.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:51 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:51.094+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:51.909 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:51 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:51.656+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:51.910 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:51 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:51.656+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:52.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:51.709+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:52.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:51 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:51.709+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:52.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:52.020+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:52.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:52.020+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:52.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:51.929+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:52.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:51.929+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:52.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:51 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:51.929+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:52.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:52 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:52.091+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:52.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:52 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:52.091+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:52.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:52 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:52.091+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:52.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:52.055+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:52.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:52.055+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:52.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:52.055+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:53.004 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:52.700+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:53.004 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:52 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:52.700+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:53.052 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:52 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:52.676+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:53.052 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:52 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:52.676+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:53.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.004+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:53.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.004+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:53.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.004+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:53.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:52.984+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:53.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:52.984+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:53.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:52.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:53.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:52.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:53.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:52 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:52.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:53.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:53 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:53.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:53.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:53 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:53.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:53.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:53 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:53.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:53.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:53 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:53.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:53.968 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:53.683+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:53.968 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:53.683+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:53.968 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:53.683+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:05:54.014 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:53 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:53.715+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:54.015 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:53 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:53.715+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:54.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:54.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:54.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:54.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:53 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:53.968+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:05:54.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:54 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:54.025+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:54.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:54 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:54.025+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:54.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:53 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:53.953+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:54.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:53 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:53.953+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:54.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:53 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:53.953+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:54.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.014+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:54.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.014+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:54.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.014+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:54.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.014+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:55.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:54.694+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:55.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:54.694+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:55.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:54.694+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:05:55.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:54.938+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:55.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:54.938+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:55.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:54.938+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:55.013 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:54 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:54.938+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:05:55.160 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:54.754+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.856793+0000 front 2024-05-08T06:05:13.856731+0000 (oldest deadline 2024-05-08T06:05:39.756621+0000) 2024-05-08T06:05:55.160 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6[58449]: 2024-05-08T06:05:54.754+0000 7fcc28438700 -1 osd.6 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.757195+0000 front 2024-05-08T06:05:19.757176+0000 (oldest deadline 2024-05-08T06:05:45.657295+0000) 2024-05-08T06:05:55.160 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.966+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:55.160 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.966+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:55.160 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.966+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:55.160 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:54 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:54.966+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:55.270 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:55 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:55.035+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:55.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:55 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:55.035+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:55.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:54 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:54.996+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:55.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:54 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:54.996+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:55.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:54 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:54.996+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:55.981 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:55.650+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:55.981 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:55.650+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:55.981 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:55.650+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:05:56.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:55.980+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:56.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:55.980+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:56.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:55.980+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:56.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:55.980+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:05:56.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:56 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:56.008+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:56.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:56 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:56.008+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:56.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:56 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:56.008+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:05:56.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:55 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:55.948+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:56.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:55 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:55.948+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:56.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:55 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:55.948+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:56.410 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:56 smithi003 podman[99877]: 2024-05-08 06:05:56.118607191 +0000 UTC m=+5.680046576 container died ae1fcecd0b0ccca226ba3fb62c1ad6a03805ec79e0261d294e2bcc7e7be969bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True) 2024-05-08T06:05:56.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:56 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:56.012+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:56.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:56 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:56.012+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:56.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:56 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:56.012+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:56.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:56 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:56.012+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:56.765 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:56 smithi003 podman[99877]: 2024-05-08 06:05:56.522924969 +0000 UTC m=+6.084364362 container cleanup ae1fcecd0b0ccca226ba3fb62c1ad6a03805ec79e0261d294e2bcc7e7be969bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:56.765 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:56 smithi003 bash[99877]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6 2024-05-08T06:05:57.005 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:56 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:56.630+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:57.006 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:56 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:56.630+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:57.006 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:56 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:56.630+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:05:57.047 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:56 smithi003 podman[99894]: 2024-05-08 06:05:56.766000593 +0000 UTC m=+0.648085678 container remove ae1fcecd0b0ccca226ba3fb62c1ad6a03805ec79e0261d294e2bcc7e7be969bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-08T06:05:57.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:57.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:57.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:57.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:57.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:57.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:57.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:57.005+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:05:57.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:57.021+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:57.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:57.021+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:57.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:57.021+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:05:57.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:56 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:56.900+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:57.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:56 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:56.900+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:57.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:56 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:56.900+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:57.323 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:57 smithi003 podman[99986]: 2024-05-08 06:05:57.092657544 +0000 UTC m=+0.018298248 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:05:57.323 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:57 smithi003 podman[99986]: 2024-05-08 06:05:57.208853113 +0000 UTC m=+0.134493805 container create ce41f5096503bf0cfaa75c45857c4ea75b81e0daaf5daeede0a031038b11250c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-08T06:05:57.323 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:57 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:57.047+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:57.323 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:57 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:57.047+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:57.323 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:57 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:57.047+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:57.323 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:57 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:57.047+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:57.660 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:57 smithi003 podman[99986]: 2024-05-08 06:05:57.453323695 +0000 UTC m=+0.378964401 container init ce41f5096503bf0cfaa75c45857c4ea75b81e0daaf5daeede0a031038b11250c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:05:57.660 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:57 smithi003 podman[99986]: 2024-05-08 06:05:57.457012535 +0000 UTC m=+0.382653232 container start ce41f5096503bf0cfaa75c45857c4ea75b81e0daaf5daeede0a031038b11250c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-05-08T06:05:57.660 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:57 smithi003 podman[99986]: 2024-05-08 06:05:57.53619392 +0000 UTC m=+0.461834614 container attach ce41f5096503bf0cfaa75c45857c4ea75b81e0daaf5daeede0a031038b11250c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-08T06:05:58.008 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:57.630+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:58.008 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:57.630+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:58.008 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:57 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:57.630+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:05:58.088 INFO:journalctl@ceph.osd.6.smithi003.stdout:May 08 06:05:57 smithi003 podman[99986]: 2024-05-08 06:05:57.766974278 +0000 UTC m=+0.692614968 container died ce41f5096503bf0cfaa75c45857c4ea75b81e0daaf5daeede0a031038b11250c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-08T06:05:58.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.008+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:58.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.008+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:58.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.008+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:58.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.008+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:05:58.270 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:58 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:58.052+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:58.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:58 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:58.052+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:58.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:58 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:58.052+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:05:58.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:57.898+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:58.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:57.898+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:58.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:57 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:57.898+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:58.370 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:58.087+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:58.370 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:58.087+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:58.370 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:58.087+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:58.370 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:58 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:58.087+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:58.392 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T06:05:58.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:05:58.433 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T06:05:58.433 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T06:05:58.433 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.7 2024-05-08T06:05:58.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 systemd[1]: Stopping Ceph osd.7 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:05:58.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:58.632+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:58.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:58.632+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:58.729 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:58.632+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:05:58.987 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:58.757+0000 7faf4f8d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:05:58.987 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:58.757+0000 7faf4f8d1700 -1 osd.7 105 *** Got signal Terminated *** 2024-05-08T06:05:58.988 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:58.757+0000 7faf4f8d1700 -1 osd.7 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:05:59.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:05:59.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:05:59.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:05:59.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:58.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:05:59.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:59 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:59.098+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:05:59.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:59 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:59.098+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:05:59.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:05:59 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:05:59.098+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:05:59.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:58 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:58.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:05:59.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:58 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:58.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:05:59.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:58 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:58.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:05:59.355 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:59.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:05:59.356 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:59.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:05:59.356 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:59.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:05:59.356 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:05:59 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:05:59.052+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:05:59.952 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:59.645+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:05:59.952 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:59.645+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:05:59.952 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:05:59.645+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:06:00.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:59.952+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:00.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:59.952+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:00.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:59.952+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:00.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:05:59 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:05:59.952+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:00.270 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:00.144+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:00.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:00.144+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:00.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:00.144+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:00.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:59 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:59.903+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:00.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:59 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:59.903+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:00.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:05:59 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:05:59.903+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:00.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:00.029+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:00.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:00.029+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:00.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:00.029+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:00.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:00 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:00.029+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:00.977 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:00.624+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:06:00.977 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:00.624+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:06:00.977 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:00.624+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:06:00.977 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:00.624+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:37.961712+0000 front 2024-05-08T06:05:37.961648+0000 (oldest deadline 2024-05-08T06:06:00.261855+0000) 2024-05-08T06:06:01.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:00.976+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:01.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:00.976+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:01.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:00.976+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:01.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:00 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:00.976+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:01.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:00.899+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:01.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:00.899+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:01.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:00 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:00.899+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:01.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:01 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:01.130+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:01.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:01 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:01.130+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:01.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:01 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:01.130+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:01.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:01.078+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:01.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:01.078+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:01.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:01.078+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:01.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:01.078+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:01.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:01 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:01.078+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:02.010 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:01 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:01.618+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:06:02.010 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:01 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:01.618+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:06:02.010 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:01 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:01.618+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:06:02.010 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:01 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:01.618+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:37.961712+0000 front 2024-05-08T06:05:37.961648+0000 (oldest deadline 2024-05-08T06:06:00.261855+0000) 2024-05-08T06:06:02.140 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:01 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:01.879+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:02.140 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:01 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:01.879+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:02.140 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:01 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:01.879+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:02.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.010+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:02.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.010+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:02.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.010+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:02.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.010+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:02.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:02.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:02.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:02.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:02.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:02.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:02.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:02.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:02.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:02 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:02.106+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:02.521 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:02 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:02.139+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:02.521 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:02 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:02.139+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:02.521 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:02 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:02.139+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:02.995 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:02.622+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:06:02.995 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:02.622+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:06:02.995 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:02.622+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:06:02.995 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:02.622+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:37.961712+0000 front 2024-05-08T06:05:37.961648+0000 (oldest deadline 2024-05-08T06:06:00.261855+0000) 2024-05-08T06:06:03.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.995+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:03.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.995+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:03.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.995+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:03.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:02 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:02.995+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:03.270 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:02 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:02.910+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:03.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:02 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:02.910+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:03.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:02 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:02.910+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:03.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:03.105+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:03.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:03.105+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:03.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:03.105+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:03.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:03.156+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:03.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:03.156+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:03.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:03.156+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:03.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:03.156+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:03.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:03 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:03.156+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:04.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:03.611+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.457566+0000 front 2024-05-08T06:05:13.457667+0000 (oldest deadline 2024-05-08T06:05:38.757940+0000) 2024-05-08T06:06:04.013 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:03.611+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.659172+0000 front 2024-05-08T06:05:21.659040+0000 (oldest deadline 2024-05-08T06:05:45.759042+0000) 2024-05-08T06:06:04.013 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:03.611+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.260268+0000 front 2024-05-08T06:05:29.260364+0000 (oldest deadline 2024-05-08T06:05:53.360269+0000) 2024-05-08T06:06:04.013 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7[52893]: 2024-05-08T06:06:03.611+0000 7faf47ebd700 -1 osd.7 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:37.961712+0000 front 2024-05-08T06:05:37.961648+0000 (oldest deadline 2024-05-08T06:06:00.261855+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:03.902+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:03.902+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:03 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:03.902+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:04.089+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:04.089+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:04.089+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:04.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:04.089+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:04.425 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:04 smithi089 podman[71929]: 2024-05-08 06:06:04.104104785 +0000 UTC m=+5.479107046 container died f37446c68e26745bb3b456b3a8764736e5faa08958eefe843821feab3551a892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T06:06:04.425 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:04 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:04.021+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:04.426 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:04 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:04.021+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:04.426 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:04 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:04.021+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:04.426 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:04 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:04.021+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:04.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:04.170+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:04.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:04.170+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:04.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:04.170+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:04.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:04.170+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:04.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:04 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:04.170+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:04.725 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:04 smithi089 podman[71929]: 2024-05-08 06:06:04.508478515 +0000 UTC m=+5.883480765 container cleanup f37446c68e26745bb3b456b3a8764736e5faa08958eefe843821feab3551a892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-08T06:06:04.725 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:04 smithi089 bash[71929]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7 2024-05-08T06:06:05.012 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:04 smithi089 podman[71957]: 2024-05-08 06:06:04.726622805 +0000 UTC m=+0.621338314 container remove f37446c68e26745bb3b456b3a8764736e5faa08958eefe843821feab3551a892 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:04.880+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:04.880+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:04.880+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:04 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:04.880+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:05.070+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:05.070+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:05.070+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:05.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:05.070+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:05.337 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:05 smithi089 podman[72048]: 2024-05-08 06:06:04.963430951 +0000 UTC m=+0.017990132 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:05.338 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:05 smithi089 podman[72048]: 2024-05-08 06:06:05.077636874 +0000 UTC m=+0.132196056 container create 9941e0c54670ef9aa5695d7f8b48a2f92976656919b397acdb4e785451cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T06:06:05.338 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:05 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:05.026+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:05.338 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:05 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:05.026+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:05.338 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:05 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:05.026+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:05.338 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:05 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:05.026+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:05.338 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:05 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:05.026+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:05.634 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:05 smithi089 podman[72048]: 2024-05-08 06:06:05.33867907 +0000 UTC m=+0.393238251 container init 9941e0c54670ef9aa5695d7f8b48a2f92976656919b397acdb4e785451cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-08T06:06:05.634 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:05 smithi089 podman[72048]: 2024-05-08 06:06:05.341856208 +0000 UTC m=+0.396415396 container start 9941e0c54670ef9aa5695d7f8b48a2f92976656919b397acdb4e785451cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-08T06:06:05.634 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:05 smithi089 podman[72048]: 2024-05-08 06:06:05.430403671 +0000 UTC m=+0.484962868 container attach 9941e0c54670ef9aa5695d7f8b48a2f92976656919b397acdb4e785451cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-08T06:06:05.659 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:05.194+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:05.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:05.194+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:05.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:05.194+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:05.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:05.194+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:05.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:05 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:05.194+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:05.916 INFO:journalctl@ceph.osd.7.smithi089.stdout:May 08 06:06:05 smithi089 podman[72048]: 2024-05-08 06:06:05.634731481 +0000 UTC m=+0.689290666 container died 9941e0c54670ef9aa5695d7f8b48a2f92976656919b397acdb4e785451cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-08T06:06:06.183 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:06 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:06.060+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:06.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:06 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:06.060+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:06.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:06 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:06.060+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:06.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:06 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:06.060+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:06.184 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:06 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:06.060+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:06.203 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T06:06:06.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:06.241 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T06:06:06.241 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T06:06:06.241 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.8 2024-05-08T06:06:06.268 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:05.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:06.268 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:05.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:06.268 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:05.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:06.268 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:05 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:05.925+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:06.269 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.027+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:06.269 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.027+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:06.269 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.027+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:06.269 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.027+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:06.521 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 systemd[1]: Stopping Ceph osd.8 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:06.659 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:06 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:06.205+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:06.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:06 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:06.205+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:06.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:06 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:06.205+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:06.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:06 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:06.205+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:06.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:06 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:06.205+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:06.889 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.539+0000 7feaf9c19700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:06.890 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.539+0000 7feaf9c19700 -1 osd.8 105 *** Got signal Terminated *** 2024-05-08T06:06:06.890 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:06.539+0000 7feaf9c19700 -1 osd.8 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:07.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:07.044+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:07.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:07.044+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:07.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:07.044+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:07.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:07.044+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:07.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:06.889+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:07.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:06.889+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:07.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:06.889+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:07.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:06 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:06.889+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:07.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:07.073+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:07.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:07.073+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:07.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:07.073+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:07.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:07.073+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:07.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:07.073+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:07.659 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:07 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:07.184+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:07.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:07 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:07.184+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:07.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:07 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:07.184+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:07.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:07 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:07.184+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:07.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:07 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:07.184+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:08.083+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:08.083+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:08.083+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:08.083+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:07.867+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:07.867+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:07.867+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:08.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:07 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:07.867+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:08.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:08.036+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:08.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:08.036+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:08.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:08.036+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:08.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:08.036+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:08.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:08 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:08.036+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:08.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:08.216+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:08.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:08.216+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:08.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:08.216+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:08.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:08.216+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:08.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:08 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:08.216+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:09.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:08.882+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:09.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:08.883+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:09.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:08.883+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:09.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:08 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:08.883+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:09.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:09.037+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:09.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:09.037+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:09.272 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:09.037+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:09.272 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:09.037+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:09.272 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:09.037+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.677886+0000 front 2024-05-08T06:05:46.677908+0000 (oldest deadline 2024-05-08T06:06:08.378067+0000) 2024-05-08T06:06:09.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:09 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:09.074+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:09.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:09 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:09.074+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:09.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:09 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:09.074+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:09.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:09 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:09.074+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:09.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:09 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:09.074+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:09.613 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:09 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:09.258+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:09.613 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:09 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:09.258+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:09.613 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:09 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:09.258+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:09.613 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:09 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:09.258+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:09.613 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:09 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:09.258+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:10.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:10.079+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:10.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:10.079+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:10.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:10.079+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:10.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:10.079+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:10.272 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:10.079+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.677886+0000 front 2024-05-08T06:05:46.677908+0000 (oldest deadline 2024-05-08T06:06:08.378067+0000) 2024-05-08T06:06:10.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:09.913+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:10.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:09.913+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:10.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:09.913+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:10.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:09.913+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:10.272 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:09 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:09.913+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:10.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.029+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:10.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.029+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:10.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.029+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:10.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.029+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:10.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.029+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:10.659 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:10 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:10.231+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:10.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:10 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:10.231+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:10.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:10 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:10.231+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:10.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:10 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:10.231+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:10.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:10 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:10.231+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:11.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.992+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:11.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.992+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:11.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.992+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:11.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.992+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:11.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.992+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:11.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:10 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:10.992+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:11.270 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:11.128+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:13.572014+0000 front 2024-05-08T06:05:13.572198+0000 (oldest deadline 2024-05-08T06:05:35.871999+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:11.128+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:19.273667+0000 front 2024-05-08T06:05:19.273702+0000 (oldest deadline 2024-05-08T06:05:44.573925+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:11.128+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:30.975432+0000 front 2024-05-08T06:05:30.975385+0000 (oldest deadline 2024-05-08T06:05:55.675537+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:11.128+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:35.676011+0000 front 2024-05-08T06:05:35.676072+0000 (oldest deadline 2024-05-08T06:06:03.276648+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8[53066]: 2024-05-08T06:06:11.128+0000 7feaf2205700 -1 osd.8 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.677886+0000 front 2024-05-08T06:05:46.677908+0000 (oldest deadline 2024-05-08T06:06:08.378067+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:10.894+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:10.894+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:10.894+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:10.894+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:11.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:10.894+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:11.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:11.232+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:11.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:11.232+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:11.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:11.232+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:11.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:11.232+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:11.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:11.232+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:11.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:11 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:11.232+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:12.148 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:11 smithi104 podman[71856]: 2024-05-08 06:06:11.881516079 +0000 UTC m=+5.447541822 container died f6019baed9f8d2527b8b1da79f094aa206463c9c76045f7c35cdb49120dba578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:12.148 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:11.891+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:12.148 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:11.891+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:12.149 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:11.891+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:12.149 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:11.891+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:12.149 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:11 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:11.891+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:12.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:11 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:11.966+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:12.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:11 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:11.966+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:12.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:11 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:11.966+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:12.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:11 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:11.966+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:12.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:11 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:11.966+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:12.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:11 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:11.966+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:12.514 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:12 smithi104 podman[71856]: 2024-05-08 06:06:12.247920447 +0000 UTC m=+5.813946195 container cleanup f6019baed9f8d2527b8b1da79f094aa206463c9c76045f7c35cdb49120dba578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:12.515 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:12 smithi104 bash[71856]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8 2024-05-08T06:06:12.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:12.198+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:12.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:12.198+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:12.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:12.198+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:12.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:12.198+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:12.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:12.198+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:12.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:12 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:12.198+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:12.771 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:12 smithi104 podman[71873]: 2024-05-08 06:06:12.515713603 +0000 UTC m=+0.634101355 container remove f6019baed9f8d2527b8b1da79f094aa206463c9c76045f7c35cdb49120dba578 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-08T06:06:13.143 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:12 smithi104 podman[71959]: 2024-05-08 06:06:12.760809919 +0000 UTC m=+0.018103962 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:13.143 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:12 smithi104 podman[71959]: 2024-05-08 06:06:12.883099731 +0000 UTC m=+0.140393764 container create af0bcc6610324d1b519046c59fbf245b5a061269b1eca05a3eb85efc008e4ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:13.143 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:12.855+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:13.143 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:12.855+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:13.143 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:12.855+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:13.143 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:12.855+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:13.144 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:12.855+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:13.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:12 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:12.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:13.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:12 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:12.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:13.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:12 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:12.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:13.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:12 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:12.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:13.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:12 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:12.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:13.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:12 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:12.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:13.433 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:13 smithi104 podman[71959]: 2024-05-08 06:06:13.143864517 +0000 UTC m=+0.401158551 container init af0bcc6610324d1b519046c59fbf245b5a061269b1eca05a3eb85efc008e4ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:06:13.433 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:13 smithi104 podman[71959]: 2024-05-08 06:06:13.147151513 +0000 UTC m=+0.404445547 container start af0bcc6610324d1b519046c59fbf245b5a061269b1eca05a3eb85efc008e4ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f) 2024-05-08T06:06:13.433 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:13 smithi104 podman[71959]: 2024-05-08 06:06:13.227077101 +0000 UTC m=+0.484371148 container attach af0bcc6610324d1b519046c59fbf245b5a061269b1eca05a3eb85efc008e4ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-08T06:06:13.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:13 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:13.185+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:13.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:13 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:13.185+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:13.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:13 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:13.185+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:13.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:13 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:13.185+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:13.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:13 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:13.185+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:13.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:13 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:13.185+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:13.770 INFO:journalctl@ceph.osd.8.smithi104.stdout:May 08 06:06:13 smithi104 podman[71959]: 2024-05-08 06:06:13.433905923 +0000 UTC m=+0.691199953 container died af0bcc6610324d1b519046c59fbf245b5a061269b1eca05a3eb85efc008e4ae1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T06:06:14.020 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:13 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:13.903+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:14.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:13 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:13.904+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:14.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:13 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:13.904+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:14.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:13 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:13.904+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:14.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:13 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:13.904+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:14.084 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T06:06:14.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:14.161 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T06:06:14.161 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T06:06:14.161 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.9 2024-05-08T06:06:14.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:13 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:13.962+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:14.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:13 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:13.962+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:14.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:13 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:13.962+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:14.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:13 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:13.962+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:14.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:13 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:13.962+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:14.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:13 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:13.962+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:14.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.141+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:14.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.141+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:14.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.141+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:14.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.141+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:14.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.141+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:14.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.141+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:14.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 systemd[1]: Stopping Ceph osd.9 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:14.909 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.483+0000 7fba53f3b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:14.910 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.483+0000 7fba53f3b700 -1 osd.9 105 *** Got signal Terminated *** 2024-05-08T06:06:14.910 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:14 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:14.483+0000 7fba53f3b700 -1 osd.9 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:15.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:14 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:14.953+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:15.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:14 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:14.953+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:15.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:14 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:14.953+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:15.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:14 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:14.953+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:15.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:14 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:14.953+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:15.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:14 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:14.953+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:15.264 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:14 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:14.860+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:15.264 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:14 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:14.860+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:15.264 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:14 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:14.860+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:15.264 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:14 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:14.860+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:15.264 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:14 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:14.860+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:15.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:15 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:15.177+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:15.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:15 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:15.177+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:15.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:15 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:15.177+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:15.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:15 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:15.177+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:15.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:15 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:15.177+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:15.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:15 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:15.177+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:16.253 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:15.955+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:16.253 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:15.955+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:16.254 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:15.955+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:16.254 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:15.955+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:16.254 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:15.955+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:16.254 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:15 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:15.955+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:16.270 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:15 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:15.826+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:16.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:15 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:15.826+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:16.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:15 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:15.826+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:16.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:15 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:15.826+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:16.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:15 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:15.826+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:16.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:16 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:16.135+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:16.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:16 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:16.135+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:16.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:16 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:16.135+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:16.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:16 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:16.135+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:16.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:16 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:16.135+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:16.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:16 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:16.135+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:17.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:16 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:16.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:17.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:16 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:16.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:17.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:16 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:16.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:17.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:16 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:16.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:17.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:16 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:16.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:17.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:16 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:16.987+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:17.270 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:16.843+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:17.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:16.843+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:17.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:16.843+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:17.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:16.843+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:17.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:16 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:16.843+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:17.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:17 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:17.097+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:17.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:17 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:17.097+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:17.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:17 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:17.097+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:17.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:17 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:17.097+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:17.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:17 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:17.097+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:17.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:17 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:17.097+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:18.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:17 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:17.838+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:18.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:17 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:17.838+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:18.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:17 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:17.838+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:18.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:17 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:17.838+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:18.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:17 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:17.838+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:18.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:18 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:18.064+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:18.409 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:18 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:18.064+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:18.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:18 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:18.064+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:18.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:18 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:18.064+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:18.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:18 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:18.064+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:18.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:18 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:18.064+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:18.512 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.019+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:18.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.019+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:18.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.019+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:18.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.019+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:18.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.019+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:18.513 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.019+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:19.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:19.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:19.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:19.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:19.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:19.264 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:19.264 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:18 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:18.970+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:19.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:18.852+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:19.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:18.852+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:19.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:18.852+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:19.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:18.852+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:19.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:18.852+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:19.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:18.852+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:19.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:19.072+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:04.675362+0000 front 2024-05-08T06:05:04.675287+0000 (oldest deadline 2024-05-08T06:05:28.175447+0000) 2024-05-08T06:06:19.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:19.072+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.978565+0000 front 2024-05-08T06:05:14.978423+0000 (oldest deadline 2024-05-08T06:05:37.878537+0000) 2024-05-08T06:06:19.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:19.072+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:21.979815+0000 front 2024-05-08T06:05:21.979839+0000 (oldest deadline 2024-05-08T06:05:48.380666+0000) 2024-05-08T06:06:19.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:19.072+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:29.481650+0000 front 2024-05-08T06:05:29.481625+0000 (oldest deadline 2024-05-08T06:05:52.981848+0000) 2024-05-08T06:06:19.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:19.072+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:38.684229+0000 front 2024-05-08T06:05:38.684374+0000 (oldest deadline 2024-05-08T06:06:00.984212+0000) 2024-05-08T06:06:19.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9[65457]: 2024-05-08T06:06:19.072+0000 7fba5052f700 -1 osd.9 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:46.285154+0000 front 2024-05-08T06:05:46.285207+0000 (oldest deadline 2024-05-08T06:06:10.985316+0000) 2024-05-08T06:06:20.160 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:19 smithi003 podman[100103]: 2024-05-08 06:06:19.85590977 +0000 UTC m=+5.487369276 container died 50577f7be600da9ce006bdaeb43d9761c1b61c4e0e1011a507bb066e6be5a4e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-08T06:06:20.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:20.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:20.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:20.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:20.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:20.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:20.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:19 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:19.981+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:20.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:19 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:19.864+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:20.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:19 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:19.864+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:20.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:19 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:19.864+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:20.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:19 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:19.864+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:20.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:19 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:19.864+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:20.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:19 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:19.864+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:20.659 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:20 smithi003 podman[100103]: 2024-05-08 06:06:20.183161773 +0000 UTC m=+5.814621278 container cleanup 50577f7be600da9ce006bdaeb43d9761c1b61c4e0e1011a507bb066e6be5a4e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:20.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:20 smithi003 bash[100103]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9 2024-05-08T06:06:20.660 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:20 smithi003 podman[100120]: 2024-05-08 06:06:20.38483244 +0000 UTC m=+0.525844990 container remove 50577f7be600da9ce006bdaeb43d9761c1b61c4e0e1011a507bb066e6be5a4e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T06:06:21.145 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:20 smithi003 podman[100209]: 2024-05-08 06:06:20.722258362 +0000 UTC m=+0.018147059 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:21.145 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:20 smithi003 podman[100209]: 2024-05-08 06:06:20.835150275 +0000 UTC m=+0.131038962 container create e6835fa983ebd049a50769442f0a1f9b6822f2a1a56f63e46a68b9a213fcc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:06:21.145 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:21 smithi003 podman[100209]: 2024-05-08 06:06:21.045703421 +0000 UTC m=+0.341592107 container init e6835fa983ebd049a50769442f0a1f9b6822f2a1a56f63e46a68b9a213fcc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:21.145 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:21 smithi003 podman[100209]: 2024-05-08 06:06:21.049800216 +0000 UTC m=+0.345688901 container start e6835fa983ebd049a50769442f0a1f9b6822f2a1a56f63e46a68b9a213fcc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:06:21.145 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:21 smithi003 podman[100209]: 2024-05-08 06:06:21.145731167 +0000 UTC m=+0.441619864 container attach e6835fa983ebd049a50769442f0a1f9b6822f2a1a56f63e46a68b9a213fcc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-08T06:06:21.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:21.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:21.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:21.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:21.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:21.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:21.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:20 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:20.932+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:21.270 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:20 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:20.851+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:21.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:20 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:20.851+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:21.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:20 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:20.851+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:21.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:20 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:20.851+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:21.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:20 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:20.851+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:21.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:20 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:20.851+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:21.410 INFO:journalctl@ceph.osd.9.smithi003.stdout:May 08 06:06:21 smithi003 podman[100209]: 2024-05-08 06:06:21.360832922 +0000 UTC m=+0.656721607 container died e6835fa983ebd049a50769442f0a1f9b6822f2a1a56f63e46a68b9a213fcc29c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f) 2024-05-08T06:06:21.935 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T06:06:21.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:21.980 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T06:06:21.980 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T06:06:21.981 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.10 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:21 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:21.948+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:22.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 systemd[1]: Stopping Ceph osd.10 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:22.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:21 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:21.829+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:22.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:21 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:21.829+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:22.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:21 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:21.829+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:22.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:21 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:21.829+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:22.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:21 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:21.829+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:22.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:21 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:21.829+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:22.762 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.269+0000 7f4a16c47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:22.763 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.269+0000 7f4a16c47700 -1 osd.10 105 *** Got signal Terminated *** 2024-05-08T06:06:22.763 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.269+0000 7f4a16c47700 -1 osd.10 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:23.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:22 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:22.898+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:23.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:22.815+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:23.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:22.815+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:23.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:22.815+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:23.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:22.815+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:23.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:22.815+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:23.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:22.815+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:24.174 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:23 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:23.816+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:24.174 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:23 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:23.816+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:24.174 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:23 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:23.816+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:24.174 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:23 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:23.816+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:24.174 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:23 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:23.816+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:24.174 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:23 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:23.816+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:24.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:24.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:24.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:24.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:24.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:24.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:24.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:23 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:23.899+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:25.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:25.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:25.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:25.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:25.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:25.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:25.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:24 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:24.896+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:25.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:24 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:24.810+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:25.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:24 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:24.810+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:25.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:24 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:24.810+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:25.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:24 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:24.810+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:25.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:24 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:24.810+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:25.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:24 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:24.810+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:26.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:26.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:26.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:26.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:26.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:26.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:26.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:25 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:25.927+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:26.270 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:25 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:25.774+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:26.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:25 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:25.774+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:26.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:25 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:25.774+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:26.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:25 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:25.774+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:26.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:25 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:25.774+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:26.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:25 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:25.774+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:27.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:26 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:26.766+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:27.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:26 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:26.766+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:27.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:26 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:26.766+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:27.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:26 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:26.766+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:27.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:26 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:26.766+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:27.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:26 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:26.766+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:27.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.287913+0000 front 2024-05-08T06:05:05.287834+0000 (oldest deadline 2024-05-08T06:05:29.987988+0000) 2024-05-08T06:06:27.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:15.288954+0000 front 2024-05-08T06:05:15.289001+0000 (oldest deadline 2024-05-08T06:05:37.589116+0000) 2024-05-08T06:06:27.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:17.589646+0000 front 2024-05-08T06:05:17.589694+0000 (oldest deadline 2024-05-08T06:05:45.790061+0000) 2024-05-08T06:06:27.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6814 osd.3 since back 2024-05-08T06:05:28.691008+0000 front 2024-05-08T06:05:28.691062+0000 (oldest deadline 2024-05-08T06:05:53.390951+0000) 2024-05-08T06:06:27.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:39.092780+0000 front 2024-05-08T06:05:39.092817+0000 (oldest deadline 2024-05-08T06:06:04.992937+0000) 2024-05-08T06:06:27.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:47.293985+0000 front 2024-05-08T06:05:47.294035+0000 (oldest deadline 2024-05-08T06:06:10.793862+0000) 2024-05-08T06:06:27.263 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:26 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10[59915]: 2024-05-08T06:06:26.926+0000 7f4a1323b700 -1 osd.10 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.095293+0000 front 2024-05-08T06:05:53.095200+0000 (oldest deadline 2024-05-08T06:06:18.394993+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:28.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:27 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:27.750+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:28.052 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:27 smithi089 podman[72152]: 2024-05-08 06:06:27.768469578 +0000 UTC m=+5.600945777 container died bb40211295edc85a8a98f8bdc902a045c9a109b2890e10629c1ebd434f5d5c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-08T06:06:28.052 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72152]: 2024-05-08 06:06:28.053456929 +0000 UTC m=+5.885933130 container cleanup bb40211295edc85a8a98f8bdc902a045c9a109b2890e10629c1ebd434f5d5c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-08T06:06:28.321 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 bash[72152]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10 2024-05-08T06:06:28.638 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72169]: 2024-05-08 06:06:28.321762312 +0000 UTC m=+0.548160768 container remove bb40211295edc85a8a98f8bdc902a045c9a109b2890e10629c1ebd434f5d5c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-05-08T06:06:28.958 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72263]: 2024-05-08 06:06:28.63952064 +0000 UTC m=+0.084095726 container create a9996d92e59f5e09d5d689ed8caff3fa9d9848cd7ab5c26c8c496691003024df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:28.958 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72263]: 2024-05-08 06:06:28.573293914 +0000 UTC m=+0.017869035 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:28.958 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72263]: 2024-05-08 06:06:28.883572613 +0000 UTC m=+0.328147704 container init a9996d92e59f5e09d5d689ed8caff3fa9d9848cd7ab5c26c8c496691003024df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T06:06:28.958 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72263]: 2024-05-08 06:06:28.886938489 +0000 UTC m=+0.331513576 container start a9996d92e59f5e09d5d689ed8caff3fa9d9848cd7ab5c26c8c496691003024df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:28.958 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:28 smithi089 podman[72263]: 2024-05-08 06:06:28.958555862 +0000 UTC m=+0.403130960 container attach a9996d92e59f5e09d5d689ed8caff3fa9d9848cd7ab5c26c8c496691003024df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-08T06:06:29.262 INFO:journalctl@ceph.osd.10.smithi089.stdout:May 08 06:06:29 smithi089 podman[72263]: 2024-05-08 06:06:29.179591648 +0000 UTC m=+0.624166737 container died a9996d92e59f5e09d5d689ed8caff3fa9d9848cd7ab5c26c8c496691003024df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T06:06:29.270 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:29.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:29.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:29.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:29.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:29.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:29.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:28 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:28.794+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:29.675 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T06:06:29.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:29.712 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T06:06:29.712 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T06:06:29.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.11 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 systemd[1]: Stopping Ceph osd.11 for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:30.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:29 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:29.789+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:30.520 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.023+0000 7fdc3fbfc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:30.521 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.023+0000 7fdc3fbfc700 -1 osd.11 105 *** Got signal Terminated *** 2024-05-08T06:06:30.521 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.023+0000 7fdc3fbfc700 -1 osd.11 105 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:31.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:30 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:30.746+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:32.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:32.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:32.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:32.022 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:32.022 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:32.022 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:32.022 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:31 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:31.728+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:33.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:32 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:32.733+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:34.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:33 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:33.741+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6806 osd.0 since back 2024-05-08T06:05:05.907245+0000 front 2024-05-08T06:05:05.907274+0000 (oldest deadline 2024-05-08T06:05:30.607346+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6804 osd.1 since back 2024-05-08T06:05:14.708388+0000 front 2024-05-08T06:05:14.708418+0000 (oldest deadline 2024-05-08T06:05:40.608594+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6804 osd.2 since back 2024-05-08T06:05:20.608984+0000 front 2024-05-08T06:05:20.609233+0000 (oldest deadline 2024-05-08T06:05:46.509020+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6812 osd.4 since back 2024-05-08T06:05:36.311478+0000 front 2024-05-08T06:05:36.311374+0000 (oldest deadline 2024-05-08T06:06:04.512159+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6812 osd.5 since back 2024-05-08T06:05:44.512628+0000 front 2024-05-08T06:05:44.512575+0000 (oldest deadline 2024-05-08T06:06:09.212680+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.3:6822 osd.6 since back 2024-05-08T06:05:53.313513+0000 front 2024-05-08T06:05:53.313611+0000 (oldest deadline 2024-05-08T06:06:18.613505+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.89:6820 osd.7 since back 2024-05-08T06:06:03.314466+0000 front 2024-05-08T06:06:03.314565+0000 (oldest deadline 2024-05-08T06:06:27.414455+0000) 2024-05-08T06:06:35.271 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:34 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11[59832]: 2024-05-08T06:06:34.776+0000 7fdc3c1f0700 -1 osd.11 105 heartbeat_check: no reply from 172.21.15.104:6820 osd.8 since back 2024-05-08T06:06:10.315390+0000 front 2024-05-08T06:06:10.315422+0000 (oldest deadline 2024-05-08T06:06:34.415499+0000) 2024-05-08T06:06:35.771 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:35 smithi104 podman[72077]: 2024-05-08 06:06:35.490434082 +0000 UTC m=+5.584208681 container died a44f00aeac321a847b3b6cd28ad0a5ab7a151215a9da39215402632aa96ab35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-08T06:06:36.106 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:35 smithi104 podman[72077]: 2024-05-08 06:06:35.855347696 +0000 UTC m=+5.949122287 container cleanup a44f00aeac321a847b3b6cd28ad0a5ab7a151215a9da39215402632aa96ab35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True) 2024-05-08T06:06:36.106 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:35 smithi104 bash[72077]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11 2024-05-08T06:06:36.423 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72092]: 2024-05-08 06:06:36.106247887 +0000 UTC m=+0.615182150 container remove a44f00aeac321a847b3b6cd28ad0a5ab7a151215a9da39215402632aa96ab35b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T06:06:36.741 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72183]: 2024-05-08 06:06:36.326652412 +0000 UTC m=+0.018012007 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T06:06:36.741 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72183]: 2024-05-08 06:06:36.448610869 +0000 UTC m=+0.139970448 container create 76a525492631904f0156d0c1ee00469b8ced102a0b7d2eccf5618807176bc831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-08T06:06:36.741 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72183]: 2024-05-08 06:06:36.659008905 +0000 UTC m=+0.350368488 container init 76a525492631904f0156d0c1ee00469b8ced102a0b7d2eccf5618807176bc831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-08T06:06:36.742 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72183]: 2024-05-08 06:06:36.661968678 +0000 UTC m=+0.353328255 container start 76a525492631904f0156d0c1ee00469b8ced102a0b7d2eccf5618807176bc831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T06:06:36.742 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72183]: 2024-05-08 06:06:36.742213182 +0000 UTC m=+0.433572770 container attach 76a525492631904f0156d0c1ee00469b8ced102a0b7d2eccf5618807176bc831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-08T06:06:37.021 INFO:journalctl@ceph.osd.11.smithi104.stdout:May 08 06:06:36 smithi104 podman[72183]: 2024-05-08 06:06:36.955122397 +0000 UTC m=+0.646481985 container died 76a525492631904f0156d0c1ee00469b8ced102a0b7d2eccf5618807176bc831 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-08T06:06:37.548 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T06:06:37.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:37.588 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T06:06:37.588 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T06:06:37.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.a 2024-05-08T06:06:37.910 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 06:06:37 smithi003 systemd[1]: Stopping Ceph mds.a for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:38.185 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 06:06:38 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a[72606]: 2024-05-08T06:06:38.025+0000 7fad89d7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:38.185 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 06:06:38 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a[72606]: 2024-05-08T06:06:38.025+0000 7fad89d7a700 -1 mds.a *** got signal Terminated *** 2024-05-08T06:06:41.408 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 06:06:41 smithi003 podman[100314]: 2024-05-08 06:06:41.144877598 +0000 UTC m=+3.351783133 container died a199896766ac07535ba30e1255ff179c2d203780471834874a70a8cadad70d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-08T06:06:41.659 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 06:06:41 smithi003 podman[100314]: 2024-05-08 06:06:41.499748021 +0000 UTC m=+3.706653556 container cleanup a199896766ac07535ba30e1255ff179c2d203780471834874a70a8cadad70d0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-08T06:06:41.660 INFO:journalctl@ceph.mds.a.smithi003.stdout:May 08 06:06:41 smithi003 bash[100314]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-a 2024-05-08T06:06:41.889 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T06:06:41.926 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:41.926 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T06:06:41.926 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T06:06:41.926 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.d 2024-05-08T06:06:42.159 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 06:06:42 smithi003 systemd[1]: Stopping Ceph mds.d for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:42.659 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 06:06:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d[71348]: 2024-05-08T06:06:42.244+0000 7efece3a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:42.660 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 06:06:42 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d[71348]: 2024-05-08T06:06:42.244+0000 7efece3a7700 -1 mds.d *** got signal Terminated *** 2024-05-08T06:06:43.660 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 06:06:43 smithi003 podman[100449]: 2024-05-08 06:06:43.395366779 +0000 UTC m=+1.243228694 container died b97c61b19939e616bbd10196de10d5561b5fca92f9e9b7847826d9b456e299ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T06:06:44.137 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 06:06:43 smithi003 podman[100449]: 2024-05-08 06:06:43.845080714 +0000 UTC m=+1.692942630 container cleanup b97c61b19939e616bbd10196de10d5561b5fca92f9e9b7847826d9b456e299ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0) 2024-05-08T06:06:44.137 INFO:journalctl@ceph.mds.d.smithi003.stdout:May 08 06:06:43 smithi003 bash[100449]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-d 2024-05-08T06:06:44.281 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T06:06:44.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:44.320 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T06:06:44.320 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T06:06:44.320 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.g 2024-05-08T06:06:44.657 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:44 smithi003 systemd[1]: Stopping Ceph mds.g for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:44.909 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g[72160]: 2024-05-08T06:06:44.656+0000 7f369a098700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:44.909 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:44 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g[72160]: 2024-05-08T06:06:44.656+0000 7f369a098700 -1 mds.g *** got signal Terminated *** 2024-05-08T06:06:48.773 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:48 smithi003 podman[100570]: 2024-05-08 06:06:48.519382503 +0000 UTC m=+3.973485132 container died 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=) 2024-05-08T06:06:49.159 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:48 smithi003 podman[100570]: 2024-05-08 06:06:48.981866214 +0000 UTC m=+4.435968840 container cleanup 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:49.160 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:48 smithi003 bash[100570]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g 2024-05-08T06:06:49.420 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T06:06:49.482 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:49 smithi003 podman[100584]: 2024-05-08 06:06:49.266591453 +0000 UTC m=+0.748188758 container remove 794a6320023fcedd28a47991b3c3a69b575b8997b536668b0e2dc7f2c675a6da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-08T06:06:49.482 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:49 smithi003 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.g.service: Deactivated successfully. 2024-05-08T06:06:49.482 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:49 smithi003 systemd[1]: Stopped Ceph mds.g for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:06:49.482 INFO:journalctl@ceph.mds.g.smithi003.stdout:May 08 06:06:49 smithi003 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.g.service: Consumed 1.138s CPU time. 2024-05-08T06:06:49.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:49.494 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T06:06:49.494 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T06:06:49.494 DEBUG:teuthology.orchestra.run.smithi003:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.j 2024-05-08T06:06:49.802 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:49 smithi003 systemd[1]: Stopping Ceph mds.j for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:50.159 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j[70887]: 2024-05-08T06:06:49.801+0000 7f21e4aa5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:50.159 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:49 smithi003 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j[70887]: 2024-05-08T06:06:49.801+0000 7f21e4aa5700 -1 mds.j *** got signal Terminated *** 2024-05-08T06:06:54.159 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:53 smithi003 podman[100690]: 2024-05-08 06:06:53.816318756 +0000 UTC m=+4.096543985 container died 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:06:54.659 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:54 smithi003 podman[100690]: 2024-05-08 06:06:54.260548236 +0000 UTC m=+4.540773472 container cleanup 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-05-08T06:06:54.660 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:54 smithi003 bash[100690]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j 2024-05-08T06:06:54.660 INFO:journalctl@ceph.mds.j.smithi003.stdout:May 08 06:06:54 smithi003 podman[100704]: 2024-05-08 06:06:54.427992875 +0000 UTC m=+0.611911468 container remove 3b7c68ae2cb4b823e766a93282134c585f37043fff1e72cbfc3536bae80a5b95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-j, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T06:06:54.689 DEBUG:teuthology.orchestra.run.smithi003:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T06:06:54.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:54.723 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T06:06:54.723 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T06:06:54.723 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b 2024-05-08T06:06:55.012 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:54 smithi089 systemd[1]: Stopping Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:55.512 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[68662]: 2024-05-08T06:06:55.021+0000 7f8f8af78700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:55.512 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:55 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b[68662]: 2024-05-08T06:06:55.021+0000 7f8f8af78700 -1 mds.b *** got signal Terminated *** 2024-05-08T06:06:57.338 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 podman[72383]: 2024-05-08 06:06:57.08451916 +0000 UTC m=+2.170059258 container died 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:57.597 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 podman[72383]: 2024-05-08 06:06:57.412957549 +0000 UTC m=+2.498497655 container cleanup 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:57.597 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 bash[72383]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b 2024-05-08T06:06:57.847 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T06:06:57.868 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 podman[72397]: 2024-05-08 06:06:57.597433922 +0000 UTC m=+0.511800855 container remove 3579e6c753c68bb6ab7b9bb4310c4ec8dccedb5e0a4c9604c8da9408d376e898 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:06:57.868 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service: Deactivated successfully. 2024-05-08T06:06:57.868 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 systemd[1]: Stopped Ceph mds.b for 708b9316-0cfd-11ef-bc97-c7b262605968. 2024-05-08T06:06:57.868 INFO:journalctl@ceph.mds.b.smithi089.stdout:May 08 06:06:57 smithi089 systemd[1]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.b.service: Consumed 1.953s CPU time. 2024-05-08T06:06:57.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:06:57.881 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T06:06:57.881 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T06:06:57.881 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.e 2024-05-08T06:06:58.200 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 06:06:57 smithi089 systemd[1]: Stopping Ceph mds.e for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:06:58.512 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 06:06:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e[64516]: 2024-05-08T06:06:58.199+0000 7f12efaba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:06:58.512 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 06:06:58 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e[64516]: 2024-05-08T06:06:58.199+0000 7f12efaba700 -1 mds.e *** got signal Terminated *** 2024-05-08T06:07:02.460 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 06:07:02 smithi089 podman[72504]: 2024-05-08 06:07:02.210118197 +0000 UTC m=+4.105456586 container died 770bed1fe3197a6742abaee6f84baf6a035e2bc4104e31f49a9cb905cf407e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T06:07:02.762 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 06:07:02 smithi089 podman[72504]: 2024-05-08 06:07:02.64211654 +0000 UTC m=+4.537454929 container cleanup 770bed1fe3197a6742abaee6f84baf6a035e2bc4104e31f49a9cb905cf407e2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-08T06:07:02.763 INFO:journalctl@ceph.mds.e.smithi089.stdout:May 08 06:07:02 smithi089 bash[72504]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-e 2024-05-08T06:07:02.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T06:07:03.000 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:03.001 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T06:07:03.001 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T06:07:03.001 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.h 2024-05-08T06:07:03.262 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 06:07:03 smithi089 systemd[1]: Stopping Ceph mds.h for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:03.762 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 06:07:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h[64826]: 2024-05-08T06:07:03.345+0000 7fcbaf85c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:03.762 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 06:07:03 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h[64826]: 2024-05-08T06:07:03.345+0000 7fcbaf85c700 -1 mds.h *** got signal Terminated *** 2024-05-08T06:07:06.904 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 06:07:06 smithi089 podman[72639]: 2024-05-08 06:07:06.64562795 +0000 UTC m=+3.426876628 container died d0ff2251bc9ed2d382e36cc90809c2b03f53883502514e34f2253dfea1bd4346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T06:07:07.262 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 06:07:06 smithi089 podman[72639]: 2024-05-08 06:07:06.996036197 +0000 UTC m=+3.777284884 container cleanup d0ff2251bc9ed2d382e36cc90809c2b03f53883502514e34f2253dfea1bd4346 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:07:07.262 INFO:journalctl@ceph.mds.h.smithi089.stdout:May 08 06:07:06 smithi089 bash[72639]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-h 2024-05-08T06:07:07.417 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T06:07:07.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:07.450 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T06:07:07.450 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T06:07:07.450 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.k 2024-05-08T06:07:08.012 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 06:07:07 smithi089 systemd[1]: Stopping Ceph mds.k for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:08.012 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 06:07:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k[64199]: 2024-05-08T06:07:07.757+0000 7ff0f7aac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:08.012 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 06:07:07 smithi089 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k[64199]: 2024-05-08T06:07:07.757+0000 7ff0f7aac700 -1 mds.k *** got signal Terminated *** 2024-05-08T06:07:10.000 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 06:07:09 smithi089 podman[72761]: 2024-05-08 06:07:09.714904628 +0000 UTC m=+2.047700107 container died 37dbc486ad7de9b9494a535d89ad8f4cde7ab94433e9fe46ac690f28180cd118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T06:07:10.262 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 06:07:10 smithi089 podman[72761]: 2024-05-08 06:07:10.108935316 +0000 UTC m=+2.441730785 container cleanup 37dbc486ad7de9b9494a535d89ad8f4cde7ab94433e9fe46ac690f28180cd118 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T06:07:10.262 INFO:journalctl@ceph.mds.k.smithi089.stdout:May 08 06:07:10 smithi089 bash[72761]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-k 2024-05-08T06:07:10.463 DEBUG:teuthology.orchestra.run.smithi089:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T06:07:10.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:10.494 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T06:07:10.495 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T06:07:10.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.c 2024-05-08T06:07:10.771 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 06:07:10 smithi104 systemd[1]: Stopping Ceph mds.c for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:11.037 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 06:07:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c[64987]: 2024-05-08T06:07:10.808+0000 7fae155cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:11.037 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 06:07:10 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c[64987]: 2024-05-08T06:07:10.809+0000 7fae155cb700 -1 mds.c *** got signal Terminated *** 2024-05-08T06:07:12.142 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 06:07:11 smithi104 podman[72287]: 2024-05-08 06:07:11.855904266 +0000 UTC m=+1.166580313 container died 505603efcfa70fe9e26dcb925d3a4c71558ecec001c5a4e2bf332b9954147095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-08T06:07:12.484 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 06:07:12 smithi104 podman[72287]: 2024-05-08 06:07:12.24200579 +0000 UTC m=+1.552681838 container cleanup 505603efcfa70fe9e26dcb925d3a4c71558ecec001c5a4e2bf332b9954147095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-08T06:07:12.484 INFO:journalctl@ceph.mds.c.smithi104.stdout:May 08 06:07:12 smithi104 bash[72287]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-c 2024-05-08T06:07:12.612 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T06:07:12.647 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:12.647 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T06:07:12.647 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T06:07:12.647 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.f 2024-05-08T06:07:12.771 INFO:journalctl@ceph.mds.f.smithi104.stdout:May 08 06:07:12 smithi104 systemd[1]: Stopping Ceph mds.f for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:13.271 INFO:journalctl@ceph.mds.f.smithi104.stdout:May 08 06:07:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-f[64674]: 2024-05-08T06:07:12.987+0000 7f5d1931c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:13.271 INFO:journalctl@ceph.mds.f.smithi104.stdout:May 08 06:07:12 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-f[64674]: 2024-05-08T06:07:12.987+0000 7f5d1931c700 -1 mds.f *** got signal Terminated *** 2024-05-08T06:07:17.271 INFO:journalctl@ceph.mds.f.smithi104.stdout:May 08 06:07:16 smithi104 podman[72421]: 2024-05-08 06:07:16.993897863 +0000 UTC m=+4.123117767 container died 6bff09764f818ce6164c47324b847e39fcfb7aac81a46f552969ed8830f046ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-08T06:07:17.595 INFO:journalctl@ceph.mds.f.smithi104.stdout:May 08 06:07:17 smithi104 podman[72421]: 2024-05-08 06:07:17.37022155 +0000 UTC m=+4.499441532 container cleanup 6bff09764f818ce6164c47324b847e39fcfb7aac81a46f552969ed8830f046ac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T06:07:17.595 INFO:journalctl@ceph.mds.f.smithi104.stdout:May 08 06:07:17 smithi104 bash[72421]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-f 2024-05-08T06:07:17.737 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T06:07:17.768 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:17.769 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T06:07:17.769 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T06:07:17.769 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.i 2024-05-08T06:07:18.271 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:07:17 smithi104 systemd[1]: Stopping Ceph mds.i for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:18.271 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:07:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: 2024-05-08T06:07:18.098+0000 7f798ba92700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:18.271 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:07:18 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i[65298]: 2024-05-08T06:07:18.098+0000 7f798ba92700 -1 mds.i *** got signal Terminated *** 2024-05-08T06:07:21.348 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:07:21 smithi104 podman[72542]: 2024-05-08 06:07:21.089376395 +0000 UTC m=+3.097338542 container died 4bf57299307a8b0203715740d9d16acb1efb09f420127248eac48ad6a95ae899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, ceph=True) 2024-05-08T06:07:21.731 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:07:21 smithi104 podman[72542]: 2024-05-08 06:07:21.447740261 +0000 UTC m=+3.455702405 container cleanup 4bf57299307a8b0203715740d9d16acb1efb09f420127248eac48ad6a95ae899 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.vendor=CentOS) 2024-05-08T06:07:21.731 INFO:journalctl@ceph.mds.i.smithi104.stdout:May 08 06:07:21 smithi104 bash[72542]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-i 2024-05-08T06:07:21.868 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T06:07:21.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:21.898 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T06:07:21.898 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T06:07:21.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.l 2024-05-08T06:07:22.020 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 06:07:21 smithi104 systemd[1]: Stopping Ceph mds.l for 708b9316-0cfd-11ef-bc97-c7b262605968... 2024-05-08T06:07:22.521 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 06:07:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-l[64368]: 2024-05-08T06:07:22.209+0000 7fdaa8314700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T06:07:22.521 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 06:07:22 smithi104 ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-l[64368]: 2024-05-08T06:07:22.209+0000 7fdaa8314700 -1 mds.l *** got signal Terminated *** 2024-05-08T06:07:26.521 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 06:07:26 smithi104 podman[72664]: 2024-05-08 06:07:26.218436241 +0000 UTC m=+4.095648511 container died 58022b2dc0d35f2e479d7017aed12cc2bd1ed5fce1f0257ca68b0d3ab7ce0fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T06:07:26.834 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 06:07:26 smithi104 podman[72664]: 2024-05-08 06:07:26.609109759 +0000 UTC m=+4.486322083 container cleanup 58022b2dc0d35f2e479d7017aed12cc2bd1ed5fce1f0257ca68b0d3ab7ce0fab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T06:07:26.834 INFO:journalctl@ceph.mds.l.smithi104.stdout:May 08 06:07:26 smithi104 bash[72664]: ceph-708b9316-0cfd-11ef-bc97-c7b262605968-mds-l 2024-05-08T06:07:26.953 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-708b9316-0cfd-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T06:07:26.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T06:07:26.983 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T06:07:26.984 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:07:27.185 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:07:29.705 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:07:29.893 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:07:32.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T06:07:32.546 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:07:34.823 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:07:34.857 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:07:34.888 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T06:07:34.914 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T06:07:34.916 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi003/crash 2024-05-08T06:07:34.917 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:07:34.960 INFO:teuthology.orchestra.run.smithi003.stderr:tar: /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:07:34.960 INFO:teuthology.orchestra.run.smithi003.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:07:34.962 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi089/crash 2024-05-08T06:07:34.963 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:07:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:tar: /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:07:35.095 INFO:teuthology.orchestra.run.smithi089.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:07:35.097 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi104/crash 2024-05-08T06:07:35.098 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash -- . 2024-05-08T06:07:35.127 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T06:07:35.127 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-05-08T06:07:35.128 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T06:07:35.128 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-08T06:07:35.168 INFO:teuthology.orchestra.run.smithi003.stdout:2024-05-08T05:50:00.000146+0000 mon.a (mon.0) 998 : cluster [WRN] application not enabled on pool 'cephfs_metadata' 2024-05-08T06:07:35.169 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-05-08T06:07:35.169 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[SEC\]' /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-08T06:07:35.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[ERR\]' /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-08T06:07:35.323 DEBUG:teuthology.orchestra.run.smithi003:> sudo egrep '\[WRN\]' /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-08T06:07:35.397 INFO:teuthology.orchestra.run.smithi003.stdout:2024-05-08T05:50:00.000146+0000 mon.a (mon.0) 998 : cluster [WRN] application not enabled on pool 'cephfs_metadata' 2024-05-08T06:07:35.398 INFO:tasks.cephadm:Compressing logs... 2024-05-08T06:07:35.398 DEBUG:teuthology.orchestra.run.smithi003:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:07:35.440 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:07:35.443 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T06:07:35.465 INFO:teuthology.orchestra.run.smithi003.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-05-08T06:07:35.465 INFO:teuthology.orchestra.run.smithi003.stderr: /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:07:35.465 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:07:35.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-08T06:07:35.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-08T06:07:35.465 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr: 87.8% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.a.log: gzip 90.2% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr: -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:07:35.469 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-08T06:07:35.474 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log: 84.3% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:07:35.474 INFO:teuthology.orchestra.run.smithi003.stderr: 94.5% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-08T06:07:35.474 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:07:35.474 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-08T06:07:35.474 INFO:teuthology.orchestra.run.smithi003.stderr: 94.6% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-08T06:07:35.482 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:: No such file or directory 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose --/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.b.log: --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-08T06:07:35.484 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.h.log: 90.3% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.e.log: 94.5% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log: 94.4% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-08T06:07:35.485 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:07:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log 2024-05-08T06:07:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-05-08T06:07:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-08T06:07:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:gzip/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-08T06:07:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T06:07:35.487 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr: 87.8%/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.z.log: -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr: 90.4% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr: 94.4% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr: 94.5% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-08T06:07:35.488 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log: 92.1% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-08T06:07:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:07:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-08T06:07:35.489 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:07:35.489 INFO:teuthology.orchestra.run.smithi104.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:07:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-08T06:07:35.491 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:07:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-08T06:07:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T06:07:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.9.log: 91.1% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:07:35.491 INFO:teuthology.orchestra.run.smithi003.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-08T06:07:35.492 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-08T06:07:35.492 INFO:teuthology.orchestra.run.smithi089.stderr: 91.0% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T06:07:35.497 INFO:teuthology.orchestra.run.smithi003.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T06:07:35.499 INFO:teuthology.orchestra.run.smithi003.stderr:/var/log/ceph/ceph-client.admin.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-08T06:07:35.583 INFO:teuthology.orchestra.run.smithi089.stderr: 93.5% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-08T06:07:35.844 INFO:teuthology.orchestra.run.smithi104.stderr: 93.1% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-08T06:07:35.845 INFO:teuthology.orchestra.run.smithi003.stderr: 90.2% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-08T06:07:35.869 INFO:teuthology.orchestra.run.smithi104.stderr: 92.0% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-08T06:07:35.886 INFO:teuthology.orchestra.run.smithi089.stderr: 92.0% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-08T06:07:36.762 INFO:teuthology.orchestra.run.smithi003.stderr: 90.4% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-08T06:07:38.339 INFO:teuthology.orchestra.run.smithi089.stderr: 92.1% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-08T06:07:38.449 INFO:teuthology.orchestra.run.smithi104.stderr: 92.3% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-08T06:07:38.540 INFO:teuthology.orchestra.run.smithi003.stderr: 92.0% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-08T06:07:39.199 INFO:teuthology.orchestra.run.smithi089.stderr: 91.7% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-08T06:07:39.393 INFO:teuthology.orchestra.run.smithi104.stderr: 91.5% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-08T06:07:39.469 INFO:teuthology.orchestra.run.smithi089.stderr: 91.7% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-08T06:07:40.299 INFO:teuthology.orchestra.run.smithi104.stderr: 91.8% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-08T06:07:40.317 INFO:teuthology.orchestra.run.smithi089.stderr: 92.2% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-08T06:07:40.319 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-05-08T06:07:40.319 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m4.862s 2024-05-08T06:07:40.319 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m15.488s 2024-05-08T06:07:40.319 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.526s 2024-05-08T06:07:40.580 INFO:teuthology.orchestra.run.smithi104.stderr: 91.9% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-08T06:07:40.582 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-05-08T06:07:40.582 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m5.125s 2024-05-08T06:07:40.582 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m17.092s 2024-05-08T06:07:40.582 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m0.564s 2024-05-08T06:07:40.758 INFO:teuthology.orchestra.run.smithi003.stderr: 91.9% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-08T06:07:41.632 INFO:teuthology.orchestra.run.smithi003.stderr: 91.7% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-08T06:07:42.691 INFO:teuthology.orchestra.run.smithi003.stderr: 91.3% -- replaced with /var/log/ceph/708b9316-0cfd-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-08T06:07:42.693 INFO:teuthology.orchestra.run.smithi003.stderr: 2024-05-08T06:07:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:real 0m7.239s 2024-05-08T06:07:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:user 0m22.733s 2024-05-08T06:07:42.693 INFO:teuthology.orchestra.run.smithi003.stderr:sys 0m0.711s 2024-05-08T06:07:42.693 INFO:tasks.cephadm:Archiving logs... 2024-05-08T06:07:42.693 DEBUG:teuthology.misc:Transferring archived files from smithi003:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi003/log 2024-05-08T06:07:42.694 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:07:46.480 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi089/log 2024-05-08T06:07:46.481 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:07:49.270 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi104/log 2024-05-08T06:07:49.271 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T06:07:52.247 INFO:tasks.cephadm:Removing cluster... 2024-05-08T06:07:52.247 DEBUG:teuthology.orchestra.run.smithi003:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --force 2024-05-08T06:07:52.450 INFO:teuthology.orchestra.run.smithi003.stdout:Deleting cluster with fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:07:52.745 DEBUG:teuthology.orchestra.run.smithi089:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --force 2024-05-08T06:07:52.936 INFO:teuthology.orchestra.run.smithi089.stdout:Deleting cluster with fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:07:53.215 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 708b9316-0cfd-11ef-bc97-c7b262605968 --force 2024-05-08T06:07:53.407 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: 708b9316-0cfd-11ef-bc97-c7b262605968 2024-05-08T06:07:53.776 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T06:07:53.776 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:07:53.798 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:07:53.814 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T06:07:53.829 INFO:tasks.cephadm:Teardown complete 2024-05-08T06:07:53.829 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T06:07:53.841 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T06:07:53.842 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:07:53.844 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:07:53.856 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T06:07:53.958 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T06:07:53.969 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T06:07:53.969 DEBUG:teuthology.orchestra.run.smithi003:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:07:53.972 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:07:53.974 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T06:07:53.986 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:07:53.987 INFO:teuthology.orchestra.run.smithi003.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:07:53.988 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-05-08T06:07:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:07:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-05-08T06:07:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +253us[ +248us] +/- 57ms 2024-05-08T06:07:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -543us[ -543us] +/- 82ms 2024-05-08T06:07:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +184us[ +184us] +/- 79ms 2024-05-08T06:07:53.990 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:07:53.992 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:07:53.992 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-05-08T06:07:53.992 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 +240us[ +239us] +/- 57ms 2024-05-08T06:07:53.992 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -548us[ -549us] +/- 82ms 2024-05-08T06:07:53.992 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +180us[ +180us] +/- 79ms 2024-05-08T06:07:53.992 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:07:53.993 INFO:teuthology.orchestra.run.smithi003.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T06:07:53.993 INFO:teuthology.orchestra.run.smithi003.stdout:=============================================================================== 2024-05-08T06:07:53.993 INFO:teuthology.orchestra.run.smithi003.stdout:^* hv01.front.sepia.ceph.com 3 8 377 171 +214us[ +277us] +/- 55ms 2024-05-08T06:07:53.994 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -616us[ -616us] +/- 82ms 2024-05-08T06:07:53.994 INFO:teuthology.orchestra.run.smithi003.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 103 +157us[ +157us] +/- 79ms 2024-05-08T06:07:53.994 INFO:teuthology.orchestra.run.smithi003.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T06:07:53.994 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T06:07:54.005 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T06:07:54.005 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T06:07:54.031 DEBUG:teuthology.orchestra.run.smithi003:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:07:54.121 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:07:54.186 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T06:07:54.250 DEBUG:teuthology.orchestra.run.smithi003:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:07:54.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:07:54.282 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:07:54.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:07:54.308 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T06:07:54.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:07:54.335 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T06:07:54.346 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T06:07:54.357 INFO:teuthology.task.internal:Duration was 2050.400319 seconds 2024-05-08T06:07:54.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T06:07:54.368 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T06:07:54.369 DEBUG:teuthology.orchestra.run.smithi003:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:07:54.371 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:07:54.373 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T06:07:54.407 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:07:54.410 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:07:54.411 INFO:teuthology.orchestra.run.smithi003.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T06:07:54.702 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T06:07:54.702 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi003.front.sepia.ceph.com 2024-05-08T06:07:54.702 DEBUG:teuthology.orchestra.run.smithi003:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:07:54.766 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-05-08T06:07:54.766 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:07:54.788 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-05-08T06:07:54.789 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T06:07:54.815 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T06:07:54.816 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:07:54.818 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:07:54.831 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T06:07:54.880 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T06:07:54.881 DEBUG:teuthology.orchestra.run.smithi003:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:07:54.888 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:07:54.896 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T06:07:55.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T06:07:55.502 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T06:07:55.502 DEBUG:teuthology.orchestra.run.smithi003:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:07:55.528 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:07:55.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T06:07:55.576 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T06:07:55.588 DEBUG:teuthology.orchestra.run.smithi003:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:07:55.591 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:07:55.594 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:07:55.612 INFO:teuthology.orchestra.run.smithi003.stdout:kernel.core_pattern = core 2024-05-08T06:07:55.614 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-05-08T06:07:55.639 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-05-08T06:07:55.655 DEBUG:teuthology.orchestra.run.smithi003:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:07:55.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:07:55.682 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:07:55.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:07:55.696 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T06:07:55.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T06:07:55.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T06:07:55.722 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T06:07:55.723 DEBUG:teuthology.misc:Transferring archived files from smithi003:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi003 2024-05-08T06:07:55.723 DEBUG:teuthology.orchestra.run.smithi003:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:07:55.776 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi089 2024-05-08T06:07:55.776 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:07:55.810 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246/remote/smithi104 2024-05-08T06:07:55.810 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T06:07:55.844 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T06:07:55.844 DEBUG:teuthology.orchestra.run.smithi003:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:07:55.847 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:07:55.849 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T06:07:55.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T06:07:55.906 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T06:07:55.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T06:07:55.918 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T06:07:55.918 DEBUG:teuthology.orchestra.run.smithi003:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:07:55.920 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:07:55.922 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T06:07:55.933 INFO:teuthology.orchestra.run.smithi003.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:07 /home/ubuntu/cephtest 2024-05-08T06:07:55.934 INFO:teuthology.orchestra.run.smithi089.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:07 /home/ubuntu/cephtest 2024-05-08T06:07:55.948 INFO:teuthology.orchestra.run.smithi104.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 06:07 /home/ubuntu/cephtest 2024-05-08T06:07:55.949 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T06:07:55.961 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T06:07:56.099 INFO:teuthology.nuke:Checking targets against current locks 2024-05-08T06:07:56.125 DEBUG:teuthology.nuke:shortname: smithi003 2024-05-08T06:07:56.125 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T06:07:56.147 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi003.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:04.761758', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/DbRa776XN7dGQLDnT/RJV2PFz++YF2U5KSitaeSOrSLTjgEhjlZHk02QWUZX72OgCn5DAi6qwVwIg8KVvVOg='} 2024-05-08T06:07:56.152 DEBUG:teuthology.nuke:shortname: smithi089 2024-05-08T06:07:56.152 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T06:07:56.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:04.763745', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4SzUhmcW/0uuurypAK7oId25cwOoiSr8oYwF88N9Wms1/1jxSiM1ZWwr3FOPlhwKHf41ZMb7d7t/gc5WDd2js='} 2024-05-08T06:07:56.177 DEBUG:teuthology.nuke:shortname: smithi104 2024-05-08T06:07:56.177 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T06:07:56.194 INFO:teuthology.orchestra.console.smithi003:Power off 2024-05-08T06:07:56.195 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T06:07:56.208 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697246', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:25:04.765566', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGetSfF2dFYoqkCPUV4FX5KrXk03nRnLMaASmDYiaDMMkoZayO+GTCpQQjaS6j+H0y5wYltMmHhBpbip4BG5ec='} 2024-05-08T06:07:56.222 DEBUG:teuthology.orchestra.console.smithi003:power off output: Chassis Power Control: Down/Off 2024-05-08T06:07:56.222 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:07:56.247 INFO:teuthology.orchestra.console.smithi089:Power off 2024-05-08T06:07:56.247 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T06:07:56.257 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2024-05-08T06:07:56.268 DEBUG:teuthology.orchestra.console.smithi089:power off output: Chassis Power Control: Down/Off 2024-05-08T06:07:56.269 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:07:56.284 INFO:teuthology.orchestra.console.smithi104:Power off 2024-05-08T06:07:56.284 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T06:07:56.293 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-05-08T06:07:56.307 DEBUG:teuthology.orchestra.console.smithi104:power off output: Chassis Power Control: Down/Off 2024-05-08T06:07:56.307 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:07:56.329 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-05-08T06:08:00.258 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:00.295 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:00.331 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:00.368 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is on 2024-05-08T06:08:00.405 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is on 2024-05-08T06:08:00.440 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-05-08T06:08:04.373 DEBUG:teuthology.orchestra.console.smithi003:pexpect command: ipmitool -H smithi003.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:04.407 DEBUG:teuthology.orchestra.console.smithi089:pexpect command: ipmitool -H smithi089.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:04.441 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:04.484 DEBUG:teuthology.orchestra.console.smithi003:check power output: Chassis Power is off 2024-05-08T06:08:04.517 DEBUG:teuthology.orchestra.console.smithi089:check power output: Chassis Power is off 2024-05-08T06:08:04.552 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is on 2024-05-08T06:08:04.585 INFO:teuthology.orchestra.console.smithi003:Power off completed 2024-05-08T06:08:04.618 INFO:teuthology.orchestra.console.smithi089:Power off completed 2024-05-08T06:08:08.554 DEBUG:teuthology.orchestra.console.smithi104:pexpect command: ipmitool -H smithi104.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T06:08:08.665 DEBUG:teuthology.orchestra.console.smithi104:check power output: Chassis Power is off 2024-05-08T06:08:08.765 INFO:teuthology.orchestra.console.smithi104:Power off completed 2024-05-08T06:08:08.870 INFO:teuthology.run:Summary data: description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsync-tester}} duration: 2050.400318622589 failure_reason: '"2024-05-08T05:50:00.000146+0000 mon.a (mon.0) 998 : cluster [WRN] application not enabled on pool ''cephfs_metadata''" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-05-08T06:08:08.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T06:08:08.969 INFO:teuthology.run:FAIL